Change Data Capture (CDC)Trigger-based CDC PatternsMedium⏱️ ~3 min

Production Reality: Overhead and Scale Limits

The Write Path Impact: Every write operation now performs extra work. At 1000 writes per second, adding triggers that insert into change tables and update indexes increases primary database CPU usage by 10 to 30 percent. Write latency grows measurably. The numbers matter for capacity planning. Without triggers, your database might handle writes at 20 milliseconds p99. With triggers performing one additional insert and a couple of index updates, p99 latency rises to 40 milliseconds or more. For write heavy workloads at tens of thousands of operations per second, this overhead becomes problematic.
Write Latency Impact
BEFORE
20 ms
AFTER
40 ms
When It Works Well: Trigger based CDC is practical for systems below roughly 2000 to 3000 writes per second per database node. Internal line of business applications, content management systems, and many Software as a Service (SaaS) products fall into this range. The added latency and CPU overhead are acceptable trade offs for the simplicity of not needing log access. Companies in regulated industries like banking and insurance have historically used this pattern. When you cannot get permissions to read transaction logs on legacy or mainframe systems, triggers provide an auditable change capture mechanism. Storage Growth: Change tables accumulate rows rapidly. At 5000 writes per second, you generate 432 million change records per day. Even with compact representations averaging 200 bytes per row, that is 86 gigabytes daily. Without proper retention policies and partition management, you hit storage limits or slow down queries scanning the change table. Production systems typically implement time based partitioning on the change table, dropping partitions older than 7 to 30 days after downstream systems have processed them. The CDC reader must maintain its offset correctly so it does not miss data during partition maintenance.
⚠️ Common Pitfall: If the CDC reader service falls behind due to downstream outages, the change table backlog grows. A 30 minute outage at 5000 writes per second means 9 million accumulated rows. Plan for backpressure scenarios with alerts on change table size and reader lag metrics.
Compared to Log Based CDC: Transaction log based CDC tools like Debezium read directly from database write ahead logs or binary logs. They capture every change with minimal overhead (typically under 5 percent CPU increase) and handle much higher throughput, often 50000 to 100000 events per second per node. However, log based CDC requires log access, which managed database services often restrict. It also couples tightly to database internals and version compatibility. Trigger based CDC trades performance for portability and control. You can filter, mask, or enrich changes at capture time using trigger logic, something log readers cannot easily do.
💡 Key Takeaways
Trigger based CDC adds 10 to 30 percent CPU overhead and increases p99 write latency from around 20 ms to 40 ms, limiting practical use to below 2000 to 3000 writes per second
At 5000 writes per second, systems generate 432 million change records daily (86 GB at 200 bytes per row), requiring time based partition management
Change table backlog grows rapidly during outages: a 30 minute downtime at 5000 writes per second accumulates 9 million rows that must be processed during recovery
Log based CDC handles 50000 to 100000 events per second with under 5 percent overhead but requires log access that managed services often restrict
Trigger based CDC offers more flexibility for per row filtering, masking, and enrichment at capture time compared to log based approaches
📌 Examples
1A SaaS application at 1500 writes per second sees database CPU rise from 45 percent to 62 percent after enabling triggers on 8 core tables. Write latency p99 increases from 18 ms to 35 ms, still acceptable for their use case
2Production team implements weekly partition rotation on change tables, keeping 14 days of history. CDC reader tracks offset in Redis with {table: orders_changes, last_sequence: 44871923, last_commit_ts: 2024-01-15T10:23:45Z} to survive partition drops
← Back to Trigger-based CDC Patterns Overview
Production Reality: Overhead and Scale Limits | Trigger-based CDC Patterns - System Overflow