# One Outage, Zero Recurrences: Retiring PostgreSQL's Hidden Capacity Limits at a Global Financial Services Firm > First, a production sequence in the EU region reached the maximum value of a 32-bit integer: 2,147,483,647. Every insert that depended on it began failing. A platform that had run flawlessly for years could suddenly accept no new rows. Not from load. Not from storage. Not from a bad deployment. A type chosen at design time had run out of numbers. # One Outage, Zero Recurrences Retiring PostgreSQL's Hidden Capacity Limits at a Global Financial Services Firm [ Learn More ]() [ Get Support ]() * **Industry:** Global financial services (risk & analytics) * **Organization:** Publicly traded, roughly $6B in annual revenue, operations on three continents * **Environment:** AWS RDS PostgreSQL, multi-region (US/EU/APAC), read replicas + logical replication * **Engagement:** Proactive DBA managed services, ongoing since 2022 * **Services:** 24×7 incident response · Capacity-limits auditing · Schema migration engineering · Performance tuning Metric | Before | After ---|---|--- EU-region insert availability | Outage: all inserts blocked by a sequence at its INT4 maximum | Restored; fleet-wide alerting on any sequence above 80% utilization; no recurrence Transaction-ID wraparound exposure | Autovacuum silently stalled for a week, XID age climbing toward the wraparound limit | Targeted VACUUM FREEZE executed; standing XID-age and autovacuum-health alerts INT4→INT8 migration risk | Single-statement rewrite of a 1.78-billion-row table ran 6+ hours before being cancelled | Batched, reviewed migration program; post-migration plan regressions diagnosed and fixed Emergency incident record | n/a | 11 emergencies over the engagement, 100% resolved; mean time-to-resolve 0.9 days, 7 of 11 same-day Results at a glance ## Get in touch Get professional, results proven, full stack support from us! [Learn More]() ## The Client The client is a global financial services firm specializing in risk and analytics. Publicly traded, roughly $6B in annual revenue, running multi-region AWS RDS PostgreSQL across the US, EU, and APAC. Their PostgreSQL estate backs the firm's risk-analytics platforms. The largest tables hold 2.2 billion and 677 million rows, and inserts arrive continuously from screening and matching workloads. When those databases cannot accept writes, risk decisions stop. ## The Challenge PostgreSQL has hard numeric ceilings, and they stay invisible until the day you hit them. The firm hit two. First, a production sequence in the EU region reached the maximum value of a 32-bit integer: 2,147,483,647. Every insert that depended on it began failing. A platform that had run flawlessly for years could suddenly accept no new rows. Not from load. Not from storage. Not from a bad deployment. A type chosen at design time had run out of numbers. Second, earlier in the engagement, autovacuum on the EU database silently stopped running for a week. With no freezing activity, the age of the oldest unfrozen transaction ID climbed steadily toward PostgreSQL's wraparound threshold, the point at which the database protects itself by refusing new transactions entirely. Nothing was alerting on either failure mode. Sequence exhaustion and XID aging were simply not on anyone's dashboard. Both are the same class of problem: capacity limits that degrade nothing along the way and then fail absolutely. ## The Solution We ran our standard arc. Diagnose, stabilize, fix the root cause, then instrument it so it cannot recur silently. * **Emergency diagnosis and stabilization.** For the wraparound near-miss, we found the tables closest to the limit with a reusable XID-age query and ran targeted VACUUM FREEZE on a live call with the client's team, pulling the database back from the threshold. For the sequence outage, we restored EU insert capability the way we have closed every emergency in this engagement: hands-on, until it was done. * **Monitoring the limits themselves.** We added alerting for any sequence above 80% utilization across the estate, plus XID-age tracking and a two-check autovacuum-health alert in the client's existing monitoring platform. That catches both the stall and its consequences. * **An INT4→INT8 migration program.** Our audit identified every near-limit sequence and 32-bit key. We reviewed the client's migration scripts before they ran: a single-statement rewrite of a 1.78-billion-row table was cancelled after six hours and replaced with a batched, resumable approach sized to production reality. * **Taming the plan regressions the fix caused.** Widening key columns changed the planner's math. A join across the firm's two largest tables stopped using its supporting indexes, driving CPU saturation and transaction failures. We diagnosed the regression and restored correct index usage, because a capacity fix that breaks query plans has only moved the outage. ## The Results The EU insert outage has not recurred anywhere in the estate. Near-limit sequences are now found by an alert, not by a failed insert. The wraparound near-miss became a standing runbook: the XID-age query and freeze procedure were reused on later autovacuum stalls in other regions without incident. Across the full engagement, all 11 emergency incidents have been resolved, with a mean time-to-resolve of 0.9 days and 7 of 11 closed the same day. The relationship has grown year over year, expanding into archiving, partitioning, and major-version upgrade work on the same platforms. ## Why it Matters Most database monitoring watches performance: CPU, latency, disk. The failures that take a region down without warning live somewhere else, in INT4 keys, sequence maxima, and transaction-ID age. These limits give no gradual signal, so auditing and alerting on them is the only defense. The outage that did happen took a day to fix. The ones that didn't cost only a dashboard. --- [View this page online](https://www.commandprompt.com/about/success-stories-case-studies/one-outage-zero-recurrences-retiring-postgresqls-hidden-capacity-limits-at-a-global-financial-services-firm/)