Combined Strategies: Production Fan Out Patterns

Scatter Gather Pattern
Scatter gather sends a request to multiple backends and aggregates responses. A search query fans out to index shards, each shard searches its portion, results merge at the coordinator. This combines concurrency (managing many in flight requests) with parallelism (shards process simultaneously).
The coordinator must handle partial results. Set timeouts per shard: if 9 of 10 shards respond in 50ms but one takes 500ms, return results from 9 shards. Degraded results beat slow results. Track which shards are consistently slow for investigation.
Map Reduce for Aggregation
Map reduce separates parallel work (map) from aggregation (reduce). Map phase runs on all nodes simultaneously, transforming data independently. Reduce phase combines results, often requiring data movement between nodes.
The reduce phase can become a bottleneck. If map produces 1TB of intermediate data that must shuffle to reduce nodes, network becomes the constraint. Design map to minimize output size. Pre aggregate in map when possible. Use combiners to reduce data before shuffle.
Work Stealing for Load Balance
Static partitioning assumes equal work per partition. Reality differs: some partitions take longer due to data skew. Work stealing lets idle workers take tasks from busy workers. Instead of one queue per worker, workers can steal from others when their queue empties.
Work stealing requires careful synchronization. The queue must support concurrent access from the owning worker and potential thieves. Double ended queues work well: owner pops from one end, thieves steal from the other. This reduces contention compared to a single shared queue.
💡 Key Insight: Production fan out combines multiple patterns. Scatter gather distributes work, timeouts handle stragglers, partial results ensure responsiveness, and work stealing balances load. No single pattern solves everything.

💡 Key Takeaways

✓Scatter gather: fan out to shards, aggregate results, handle partial responses on timeout

✓Map reduce: parallel map phase, aggregating reduce phase that can bottleneck on data shuffle

✓Combiners pre aggregate in map phase to reduce data movement to reduce phase

✓Work stealing balances load when partitions have unequal work

✓Production systems combine patterns: scatter gather plus timeouts plus partial results

📌 Interview Tips

1When designing search, explain scatter gather: coordinator fans out to index shards, each returns top K results, coordinator merges and re ranks

2For aggregation queries, describe map reduce and why shuffle is expensive. Mention combiners as optimization

3If asked about load balancing, explain work stealing: idle workers take from busy workers using double ended queues to reduce contention

← Back to Concurrency vs Parallelism Overview