Hierarchical Caching and Origin Shield Architecture

Multi Tier CDN Architecture

Modern CDNs implement hierarchical caching with multiple tiers. Hundreds of edge PoPs (Points of Presence) serve end users, but cache misses do not go directly to origin. Instead, they route through a smaller set of regional parent caches (often called Origin Shield or mid tier). When edge misses, it fetches from its assigned parent; only if parent also misses does request reach origin. This dramatically raises effective hit ratio because popular objects fetched once into a parent can populate dozens of downstream edges without additional origin requests.

Mathematical Benefit

Consider a video segment requested from 100 edge PoPs. Without hierarchy, 100 origin requests. With parent tier where each parent aggregates 10 edges, only 10 parent requests to origin. If parents share data between themselves, possibly just 1 origin request total. Typical deployments see 50-90% origin request reduction depending on content popularity distribution. Request coalescing at parent tier collapses concurrent misses for same cache key into single origin fetch, preventing stampede scenarios where thousands of requests would otherwise overwhelm the origin.

Origin Shield Placement

Place parent caches geographically close to your origin to minimize fetch latency. Parent to origin latency adds to cache miss penalty for all downstream edges. Typical parent latency: 20-50ms when near origin vs 100-200ms for direct edge to origin across continents. The 50-150ms savings per miss compounds across all edge misses, significantly improving user experience.

Trade offs

Additional layer adds latency (5-20ms) on cache hits due to extra hop. Only use when origin protection benefits outweigh hit latency cost. Most beneficial for high fan out: same content served to many regions (video, images, static assets). Less beneficial for highly personalized content where each user sees unique data.