Composition Patterns: Strategy, Decorator, and Layered Policies at Scale

The Strategy pattern encapsulates policy choices (backoff algorithms, load balancing rules, cache eviction policies) as interchangeable components selected at runtime. At Meta and Google, request handling layers select strategies per call or per tenant to respect latency and error budgets. For example, a high priority request might use a shorter timeout (50 milliseconds) and aggressive retry, while a batch job uses a longer timeout (500 milliseconds) with exponential backoff. This per call tuning is impossible with inheritance, where behavior is fixed at compile time.

The Decorator or pipeline pattern layers cross cutting concerns around core logic in a strict, documented order. A typical production stack: deadline enforcement, then bulkhead isolation, then retry logic, then circuit breaker, then metrics, then tracing, finally the actual Remote Procedure Call (RPC). Order matters critically: placing retry after timeout amplifies load via retry storms when timeouts trigger retries that also time out, creating cascading failures. Placing retry before timeout masks slow dependencies, shifting tail latency to callers. Google's "The Tail at Scale" paper highlights how fan out inflates p99 latency; properly ordered composable timeouts, hedging, and load shedding policies mitigate this without inheritance.

Define time and attempt budgets per layer to prevent retry storms and tail amplification. For a p99 end to end target of 300 milliseconds with two fan out branches of up to five calls each, allocate less than or equal to 10 to 20 milliseconds per internal hop including policy overhead. Measure per layer contribution with structured timing; a common production failure is placing metrics or tracing outside of retries and timeouts, hiding where time is actually spent. Emit timing and error counters at every composed layer so you can pinpoint which decorator adds latency.

Composition introduces delegation overhead. Each layer adds dispatch cost, and excessive decorator stacking (more than 6 layers on hot paths) can add 5 to 20 milliseconds of overhead on a 100 to 300 millisecond p99 path. Limit depth, use coarse grained components on hot paths, and stabilize call sites to enable Just In Time (JIT) inlining. Avoid megamorphic dispatch by prebinding strategies per pool or tenant rather than swapping implementations frequently at runtime.

💡 Key Takeaways

•Strategy pattern enables per call or per tenant policy selection at runtime (50 millisecond timeout for high priority, 500 millisecond for batch) impossible with compile time inheritance

•Decorator layer order is critical: retry after timeout causes retry storms; retry before timeout masks slow dependencies and shifts tail latency to callers; correct order prevents cascading failures

•Allocate strict per layer budgets: for p99 of 300 milliseconds with fan out, allocate less than or equal to 10 to 20 milliseconds per hop including policy overhead; measure each layer with structured timing to avoid hidden costs

•Excessive decorator depth (more than 6 layers on hot paths) adds 5 to 20 milliseconds overhead; limit stack depth and use coarse grained components to keep within latency budgets

•Emit timing and error counters at every layer; common failure is placing metrics outside retries or timeouts, hiding where time is spent and causing blind spots in production debugging

📌 Examples

A Google service with p99 target of 300 milliseconds allocates 50 milliseconds deadline, 10 milliseconds for bulkhead, 20 milliseconds retry budget (2 attempts), 5 milliseconds circuit breaker check, 5 milliseconds metrics, leaving 210 milliseconds for actual work

Meta newsfeed rendering composes small React components with strict 16.7 millisecond per frame budget (60 frames per second); composition enables partial tree updates, avoiding full hierarchy invalidation on every state change

Amazon service places retry after circuit breaker but before timeout; when circuit opens due to 5 consecutive failures, retries are skipped, preventing retry storms that would amplify load 3 to 5 times

← Back to Inheritance & Composition Overview