How Exactly-Once Processing Works

The Fundamental Approach: Exactly-once processing combines three core techniques to handle duplicate message deliveries correctly. First is idempotency, where replaying an operation multiple times produces the same final state as executing it once. Second is atomicity, ensuring that reading input, updating state, and writing output happen as a single unit. Third is durable checkpoints, saving your progress so recovery knows exactly where to resume. These three pieces work together in a specific pattern that repeats continuously during normal operation. Why This Works: The magic is in the atomicity of step 4. When a failure happens, either the entire checkpoint committed (state and outputs are both saved and visible) or it did not (neither are visible). There is no middle ground where outputs went out but state was lost, or vice versa. On recovery, you replay from the last successful checkpoint. Any messages processed after that checkpoint but before the failure get reprocessed. But because outputs from that partial work were never committed, the external world never saw them. Reprocessing produces the same outputs again, which now get committed successfully. The final result is identical to processing each message exactly once. Real System Example: Apache Flink implements this pattern at massive scale. A Flink job processing 500,000 events per second might checkpoint every 5 seconds. Each checkpoint writes operator state (perhaps 2 GB of aggregated data) to S3 and coordinates with output sinks using two phase commit semantics. If a task manager crashes at second 7, Flink restores the second 5 checkpoint and replays those 2 seconds of input, roughly 1 million events. Because the partial outputs from seconds 5 to 7 were never committed, downstream systems never saw them. The replay produces identical outputs that now commit successfully.