How Event Time Processing Works at Scale

The Business Reality: Companies processing 5 to 10 million events per second globally, like video streaming services or ride sharing platforms, cannot ignore event time. Every event carries an event time timestamp indicating when playback buffered or when a rider opened the app. These events flow from mobile clients and backend services into a central message bus.

Due to network jitter and client buffering, events from the same user session might arrive tens of seconds apart. This is not an edge case, it is the normal operating condition.

The Core Mechanism: Stream processors running at hundreds of thousands of records per second per task must compute aggregates like "plays per title per minute by region" or "95th percentile startup time per ISP per 5 minutes." For these analytics to be meaningful, they must be computed based on event time, not processing time. Otherwise, your sliding or hourly windows do not correspond to human time, and you cannot reliably correlate metrics with incidents or marketing campaigns.

1
Timestamp Assignment: Extract event time from payload. Validate against server receipt time to catch extreme clock skew (for example, if client time is off by 10 minutes).
2
Window Assignment: Place event into windows based on event time boundaries (for example, "00:00 to 00:04 UTC"), regardless of when the event arrives at the processor.
3
Watermark Tracking: Maintain a watermark that says "I have seen all events up to time T, except possibly a small fraction of late arrivals." When the watermark passes a window end, emit final results.
Real World Impact: Netflix, Uber, and Meta use event time semantics for analytics, experimentation, and billing. They still monitor processing time for operational health. A monitoring alert might trigger if "events processed in the last 60 seconds drops below threshold" (processing time), while a business dashboard displays "daily active users by event time."

System Performance Targets
500 ms
P50 LATENCY
3 sec
P99 LATENCY
The Reprocessing Advantage: If you replay 24 hours of data from the log after a bug fix, processing time is completely different, but event time is the same. If your logic is anchored in event time, recomputing yesterday's metrics produces identical results. This is critical for auditability and finance grade reporting where you need reproducible outputs.

💡 Key Takeaways

✓Event time processing enables accurate analytics in the presence of out of order and late events by grouping events based on when they actually occurred

✓Systems processing millions of events per second maintain both low latency (median 500 milliseconds, 99th percentile under 3 seconds) and high throughput using event time semantics

✓Companies separate concerns: event time for business correctness and dashboards, processing time for operational monitoring and system health alerts

✓Reprocessing or backfills produce identical results with event time anchoring because event time is immutable, unlike processing time which changes on every replay

📌 Examples

1A video streaming service computes "95th percentile startup time per ISP per 5 minutes" using event time windows. During a network incident causing 2 minute delays, metrics remain accurate because events are assigned to the correct 5 minute buckets based on when playback actually started.

2When replaying 24 hours of data after fixing a bug, event time based aggregates produce the exact same "revenue per hour" numbers as the original run, ensuring financial reports can be audited and corrected without discrepancies.

← Back to Event Time vs Processing Time Overview