Every Emergency Resolved, None Caused by PostgreSQL

• Industry: Online travel: consumer vacation-rental marketplace
• Organization: A leading online vacation-rental marketplace connecting owners and travelers, with pronounced seasonal booking peaks in January and Q4
• Environment: Fully self-hosted: a four-node LXD container cluster on Ceph storage, PostgreSQL, pglogical/Bucardo replication, pgBackRest backups to S3, dedicated 10 Gbps replication network
• Engagement: Proactive managed PostgreSQL services
• Services: Full-stack platform operations · Backup & recovery engineering · Major-version upgrades · 24×7 incident response

A leading online vacation-rental marketplace has run its business on PostgreSQL since 2011. Unusually for a consumer web property of its traffic, it runs the entire stack itself: its own hardware in a colocation datacenter, its own load balancers, storage, and database cluster. Every listing, booking, and account lives in that database. There is no cloud provider to absorb a failure. When something breaks anywhere between the disk and the connection slot, someone has to own it end to end.

For fifteen years, that someone has been us.

Self-hosting means every failure mode belongs to you. Over the engagement the platform absorbed the full catalog: a failed RAID-controller battery that took a database server down, direct-attached storage throwing hardware errors for a week, a Ceph RBD volume stuck mid-delete, swap exhaustion on a storage node, and a runaway application process that inflated PostgreSQL logging from roughly 1.4 GB a day to 240 GB in under twelve hours, filling the root filesystem the same day. All of it against a seasonal travel business whose January and holiday-quarter traffic spikes forgive nothing.

Meanwhile the platform was aging out from under the workload. PostgreSQL running past end-of-life on bare metal with DRBD and NFS. Backups on legacy WAL-E tooling. Datacenter bandwidth overages recurring every month.

Our approach has been the same for fifteen years: own the whole stack, and treat recovery as the product.

• Zero-downtime replatforming. We executed a full datacenter migration with cascading replication and a switchover, no outage window. The client's engineering team's verdict on the cutover: it went perfectly.
• Container-and-Ceph modernization. We rebuilt the bare-metal DRBD/NFS platform as LXD containers on a four-node Ceph cluster, with a dedicated 10 Gbps replication network and a golden container image that registers its own DNS and monitoring. As the storage layer's failure modes surfaced (stuck RBD volumes, undersized container profiles, swap pressure), we engineered them out. We completed this with a CephFS cutover that retired the legacy NFS service entirely, replicated four ways like the rest of the cluster: 3.4 times the write throughput and 36 times the read throughput of the NFS service it replaced, executed in a single planned 30-minute window.
• Major-version escape from EOL. Postgres de-risked by a benchmarked side-by-side test environment before cutover, with security minor releases applied on cadence since.
• Recovery engineering, continuously exercised. We rebuilt backups from WAL-E to incremental, bandwidth-throttled pgBackRest with continuous WAL archiving to S3. That ended the monthly overages outright and reclaimed 8.7 TB of stale legacy backups. Restore procedures are tested and documented, not assumed. When the client needed to investigate historical table data, we caught the required full backup one scheduled prune away from deletion and extended retention on the spot, preserving the recovery point until diagnosis was complete.
• Root-cause discipline. We traced the 240 GB log explosion same-day to an application UPDATE writing oversized binary data. We traced a query stall to stale planner statistics: five seconds to milliseconds after ANALYZE. We traced a backup failure all the way down to IPv6 DNS resolution-order behavior in glibc.

Across 306 tickets and fifteen years, every emergency-priority incident was resolved, three of four the same day, and not one was caused by PostgreSQL itself. Each traced to the application, network, or load-balancer tier while the database kept serving. The critical tier overall (emergency plus high priority) stands at 98% resolved.

The durable outcomes compound: a modern container-and-Ceph platform where end-of-life bare metal once stood, a supported PostgreSQL major version, backups that are cheaper, faster, and provably restorable, and monitoring and automation that retire toil rather than defer it. Fifteen years in, the relationship keeps expanding on the same trajectory. Disaster-recovery and cloud-failover engineering are now in flight.

"It went perfectly." The client's engineering team, after the zero-downtime datacenter migration.

Managed cloud databases outsource recovery to someone else's runbook. When you self-host, recovery expertise is the product, and it cannot be bolted on during an incident. Fifteen years of that discipline buys platform migrations without downtime, backups that are exercised rather than hoped about, and a database tier that is never the reason the site is down.

A backup you have never restored is a hope. This client has never had to hope.