Audit Data Grows Forever.

• Industry: Financial technology (cloud-native core banking)
• Organization: A cloud-native core-banking platform serving U.S. financial institutions, one database per bank
• Environment: AWS RDS PostgreSQL, one database per bank, Multi-AZ, logical replication
• Engagement: Embedded DBA engineering and migration support, 2019 to 2024
• Services: Partitioning and archival engineering · Major-version upgrades · Logical replication · 24×7 incident response

The client runs a cloud-native core-banking platform for U.S. financial institutions. Every bank on the platform gets its own PostgreSQL database on AWS RDS. That database is the ledger: it posts the transactions, holds the balances, and answers the API calls that banking apps make all day. Regulators require the platform to keep a record of every API request and response it handles.

Keep every API request and response, forever, in the same database that posts transactions. That is what the compliance requirement amounted to in practice. The audit tables grew without bound and were swallowing the OLTP databases. Backups got longer. Storage costs climbed on every bank's instance. The tables the regulator cares about most were becoming the tables the database could least afford to carry.

And the fleet was aging. Every bank was on soon to be EOL PostgreSQL version, headed toward end of life, and pg_upgrade was not an option on the client's terms. These are banks. Downtime windows are negotiated, short, and per institution. Whatever we built had to work one database at a time, dozens of times, without drama.

We started where the growth was: the audit tables.

• Partitioning on the key the platform already had. The platform's primary keys were a homegrown GUID that encodes a timestamp. We designed native range partitioning on that key, so every day of audit traffic landed in its own partition, with no schema rewrite of the application's identifiers.
• A day-by-day archive pipeline to S3. Old partitions are exported as day-sized chunks, each with a manifest carrying the table's schema, so any archived day can be restored programmatically on demand. Scheduling runs through pg_timetable. The working retention agreed with the client's developers: 90 days online, everything older in S3.
• A careful production switch. Converting live audit tables cannot block live banking. We backfilled the partitioned tables with a throttled, resumable script, tested the pacing against the nightly ETL windows, then met with the client's DBA to perform the final switchover in a scheduled session. It went smoothly, and the bloated originals could finally be dropped.
• A scripted per-bank fleet migration, by logical replication. With the audit data bounded, the databases were small enough to move. We built and tested the runbook, including a technique that converts an existing binary replica into a logical replica, skipping the expensive initial copy entirely. Each bank cut over in a short scheduled window. For the largest bank we split the rollout into two steps to keep each outage small.
• Rescuing the largest migration. The largest bank's logical replication stalled with a 2TB backlog of unprocessed WAL. We diagnosed the timeout settings, explained the initial-copy semantics that make logical lag look nothing like binary lag, and identified the real ceiling: RDS write throughput tops out near 250MB per second even with provisioned storage, so two sync workers saturate it and more just burn CPU. Tuned to that reality, the subscriber caught up and the migration completed.

The engagement was not only projects. When a SAN failure at AWS throttled disk IO and took a bank fully offline at 5:25 PM, we were on it by 5:28 PM and restored service by failing over to the Multi-AZ standby.

The audit tables stopped being a threat to the OLTP workload. Online retention is bounded at 90 days per bank, the regulatory record lives durably in S3, and any archived day can be brought back with its manifest. The entire per-bank fleet moved from PostgreSQL an about to be EOL to supported version through one scripted, repeatable logical-replication process, including a two-step rollout for the largest institution and a recovery from a 2TB WAL backlog along the way. And when the platform had its worst day, a full customer outage from a storage failure, the gap between their report and our response was three minutes.