thundering herd pattern
A cached item expires. Every request that wanted it now hits the origin at the same time. The origin gets crushed by a sudden synchronized spike instead of normal traffic.
A cached item expires. Every request that wanted it now hits the origin at the same time. The origin gets crushed by a sudden synchronized spike instead of normal traffic.
symptoms
- sudden origin spike right after a cache TTL boundary
- tail latency spikes correlated with cache miss events
- load on origin much higher than steady-state
causes
- popular cache key with hard TTL
- no single-flight / coalescing on misses
- synchronized client behavior (cron, deploys)
fixes
- single-flight (one origin request per key, others wait)
- stale-while-revalidate
- randomized TTLs
- request coalescing
you might say
- dogpile on the cache miss
- everyone hit it at once
- the cache key just expired and the world ended