What is Timestamp-based CDC?

Timestamp based CDC solves a fundamental data engineering problem: how do you keep a data warehouse, search index, or cache synchronized with your production database without overwhelming it? The Core Problem: Imagine you have a 2 TB e-commerce database with 500 million product and order records. Your analytics team needs fresh data every few minutes for dashboards. Copying the entire database repeatedly is impossible. Scanning 500 million rows every 5 minutes would crush your production database and take hours to complete. How Timestamp Based CDC Works: Every table includes a timestamp column like updated_at or last_modified. Your database automatically updates this timestamp whenever a row is created or modified. The CDC system maintains a "watermark" representing the last time it checked for changes. On each run (typically every 30 seconds to 5 minutes), the CDC job queries: "Give me all rows where updated_at is greater than my watermark." It might find only 50,000 changed rows instead of scanning all 500 million. After processing those changes, it updates the watermark to the newest timestamp it saw. What Makes It Different: This is a pull based, polling approach. The CDC system periodically asks "what changed?" rather than listening to a continuous stream of database events. It works with any database that supports timestamp columns, including systems that do not expose transaction logs. This simplicity makes it the most accessible form of CDC, though it comes with important limitations around latency and correctness that we will explore in later cards.