From Zero Backups

• Industry: Healthcare technology (hospice and palliative-care EMR)
• Organization: The hospice and palliative-care division of a leading U.S. post-acute-care technology company
• Environment: Self-managed PostgreSQL in a colocation datacenter, migrating to Google Cloud (Cloud SQL, GKE, PgBouncer)
• Engagement: Proactive DBA managed services,
• Services: Backup and recovery · Health assessment · Migration support · Disaster recovery planning

The client runs the electronic medical record for hospice and palliative care providers across the United States. Nurses chart against this database at the bedside. Medication records, care plans, and compliance audit trails all live in PostgreSQL, and all of it is PHI under HIPAA. When they engaged us, the production estate still ran in a colocation datacenter, months away from a planned migration to Google Cloud.

The client told us something most companies never say out loud: assume we have no backups. They asked us to prove them wrong.

We couldn't. We assessed the colocation estate and found one host running five PostgreSQL clusters across three different major versions, each with a different backup configuration, and not one of them working. The most recent local copies were stale by months. On the warehouse cluster there was a large WAL archive with no base backup behind it, which is a pile of receipts for a purchase that was never made. On the core EMR server, every backup was failing, and a cleanup script was deleting WAL files after 14 days whether they had been archived or not.

The worst of it sat on one compliance-critical database. WAL archiving was broken both locally and to its remote target. 135 GiB of unarchived WAL had piled up, the archive volume could not hold it, and no base backup existed anywhere we could find. At the observed WAL rate, the data volume was about 48 days from full. Not a hypothetical. A date on a calendar.

We built the safety net first and argued about architecture later.

• We inventoried every cluster's backup and archiving state and documented it, so the gap was visible instead of assumed.
• We implemented pgBackRest across the estate, archiving WAL and shipping compressed backups to Google Cloud Storage buckets, out of the colocation facility entirely. We sequenced the rollout by risk: the compliance-critical database first, the core EMR last only because it needed access and disk planning, not because it could wait.
• We drained the unarchived WAL backlog once real backups existed to replace it, and wired backup status into the client's monitoring so a silent failure could never accumulate for months again.
• The migration itself was led by the client's cloud partner. Our job was the net under it. We reviewed their zero-downtime migration documentation with multiple engineers, consulted on memory and parallelism settings in the final hours before cutover, and kept an engineer on call through the November migration weekend. The cutover completed without needing us. That is the goal.
• After cutover we stayed on: we reviewed the PgBouncer deployment fronting the production EMR on Cloud SQL, backed the change from one pod to three, monitored the deploy, and ran proactive reviews of long-running and heavyweight transactions.
• We moved from safety net to fire drill: disaster recovery runbooks for backup identification and restoration, tested against a dedicated DR project.

Every production cluster went from zero restorable backups to pgBackRest backups in cloud storage, verified and monitored. The disk that had 48 days left never filled. The EMR moved to GCP without downtime, and the client kept us on afterward for pooling, transaction hygiene, and disaster recovery planning.