single-flight pattern
When N requests arrive simultaneously for the same uncached key, only one of them goes to the origin; the rest wait for that one's result. Prevents cache stampedes (and the thundering-herd variant on cache misses) without changing the cache TTL or behavior.
When N requests arrive simultaneously for the same uncached key, only one of them goes to the origin; the rest wait for that one's result. Prevents cache stampedes (and the thundering-herd variant on cache misses) without changing the cache TTL or behavior.
symptoms
- origin spikes on cache TTL boundaries
- thundering herd on popular keys
- identical concurrent computations wasting CPU
causes
- cache miss + many parallel readers
- no deduplication on the miss path
fixes
- Go's singleflight package or equivalent
- mutex per key during origin fetch
- all waiters share the result
you might say
- single-flight
- coalesce the misses
- one of them wins, others wait