Caching • Cache Patterns (Aside, Through, Back)Medium⏱️ ~3 min
Cache Aside Pattern: Application Controlled Cache Population
Cache aside, also known as lazy loading, places the application in full control of both cache population and invalidation. On reads, the application first checks the cache for the requested key. On a cache miss, it loads data from the source of truth (typically a database), populates the cache with a Time To Live (TTL), and returns the data to the caller. On writes, the application writes directly to the source of truth first, then invalidates (typically by deleting) the related cache keys to prevent serving stale data. This pattern is the default choice for read heavy workloads and provides fine grained control over what gets cached.
Meta's Memcache deployment exemplifies cache aside at massive scale. Their infrastructure uses a two tier architecture: L1 per host cache plus L2 distributed cache across thousands of servers. Published research shows sub millisecond latencies at the 95th percentile for cache hits, with hit ratios often exceeding 90% across web workloads. This offloads the vast majority of reads from MySQL shards, which would otherwise face cache miss penalties of a few milliseconds for intra datacenter database hits. Meta employs lease tokens to prevent thundering herds, where many clients simultaneously miss the same hot key and overwhelm the database. They use delete on write instead of update cache to avoid race conditions where concurrent readers might cache stale values.
The trade off with cache aside is increased application complexity and the risk of cache stampedes if not properly mitigated. Applications must implement careful concurrency controls, including single flight or lease mechanisms to ensure only one loader fetches on a miss. Multiget operations reduce network round trips when fetching multiple keys. TTL jitter (randomizing expiry times by plus or minus 10 to 20 percent) prevents synchronized expiry that would cause simultaneous misses. The pattern works exceptionally well with denormalized key designs where you cache precomputed views, and allows selective caching decisions per entity type based on access patterns.
💡 Key Takeaways
•Application owns all cache decisions: check cache on read, load and populate on miss, write to database then delete cache keys on writes
•Meta achieves sub millisecond p95 latency with over 90% hit ratios using two tier cache (L1 per host, L2 distributed), serving millions of operations per second per region
•Delete on write pattern avoids race conditions where update cache could overwrite with stale data from concurrent readers
•Requires thundering herd protection via lease tokens or single flight to prevent multiple loaders overwhelming the database on simultaneous misses
•Works best for read dominated workloads (90:10 read to write ratio or higher) with denormalized key designs and selective caching
•Memory access takes 0.1 to 1 millisecond end to end, while local SSD backed database reads take 1 to 5 milliseconds, making high hit ratios essential given RAM is 10 to 100 times more expensive per GB than SSD
📌 Examples
Meta Memcache implementation: Application checks L1 cache (per host), then L2 distributed cache (thousands of servers), then MySQL on miss. Uses lease tokens to ensure single loader, negative caching for misses, and multiget to batch requests and reduce round trips.
Netflix EVCache for metadata: Client library provides cache aside semantics with replication across Availability Zones. Sub millisecond hit latency within AZ, cross AZ adds 1 to 2 milliseconds. Handles millions of operations per second with replication factor of 2 to 3 for AZ failure tolerance.
Typical read implementation: value = cache.get(key); if (value == null) { lease = acquireLease(key); if (lease.acquired) { value = database.query(key); cache.set(key, value, ttl + randomJitter()); releaseLease(key); } else { waitForLeaseHolder(); value = cache.get(key); } } return value;