Micro-batching vs Stream vs Batch: Choosing the Right Pattern

The Fundamental Trade-off: Choosing between micro-batching, pure streaming, and traditional batch processing is about balancing latency, consistency, complexity, and cost. Each pattern occupies a different point in this design space, and understanding when to use each is critical for system design interviews. Latency Comparison: Compared to hourly batch processing, micro-batching dramatically reduces latency. Instead of p99 latencies measured in hours, you get p99 in tens of seconds. For a monitoring dashboard, this transforms the user experience from stale hourly snapshots to near real-time updates. Compared to true streaming that processes each event as it arrives, micro-batching trades higher latency for simplicity and lower cost. Pure streaming systems like Flink or Kafka Streams achieve 10 to 100 millisecond end-to-end latencies by processing events individually. Micro-batching with 5 second intervals delivers 5 to 20 second latencies. If your use case needs sub-second response for fraud detection or high frequency trading, micro-batching is too slow. But for analytics, monitoring, or batch ML feature updates, the extra seconds don't matter. Complexity and Cost: Micro-batching is usually simpler and cheaper than pure streaming. The system has fewer coordination events, fewer state updates per second, and better opportunities for vectorized computation and I/O batching. Processing 50,000 events together reduces per-record overhead compared to 50,000 individual processing steps. Consider CPU usage: a pure streaming job might update state and checkpoint progress thousands of times per second. A micro-batch job does this once per batch, reducing coordination overhead by 1,000x to 10,000x. This can cut compute costs by 30 to 50 percent for workloads that can tolerate the latency increase. Semantics and Fault Tolerance: Micro-batching makes exactly-once semantics easier to implement. You can design sinks to be idempotent per batch or transactional per batch, using batch identifiers and source offset ranges. Pure streaming often needs more complex per-record idempotency or distributed transaction protocols. However, streaming engines with fine-grained checkpoints may react to traffic spikes faster. If ingestion suddenly doubles, a streaming system can start processing new events within milliseconds. A micro-batch system might not start the next batch for several seconds, during which backlog accumulates. Decision Framework: Use micro-batching when your Service Level Objective (SLO) is measured in seconds to low tens of seconds, when your team is comfortable with batch style programming (SQL, DataFrame operations), and when simplicity and cost matter more than absolute latency. Prefer pure streaming when you need millisecond latencies (fraud detection, real-time bidding, high frequency trading), when you need continuous per-record state updates (complex event processing with immediate triggers), or when you have very fine-grained windowing requirements (sliding windows of 100 milliseconds). Stick with traditional macro batch when your latency tolerance is minutes to hours (daily reports, training ML models on historical data), and when your main constraint is cost and operational simplicity.