token bucket pattern
Rate-limiting algorithm. A bucket holds up to N tokens; tokens are added at a fixed refill rate up to that cap. Each request consumes a token; if the bucket is empty, the request is rejected (or queued). The bucket size sets the burst tolerance; the refill rate sets the sustained rate.
Rate-limiting algorithm. A bucket holds up to N tokens; tokens are added at a fixed refill rate up to that cap. Each request consumes a token; if the bucket is empty, the request is rejected (or queued). The bucket size sets the burst tolerance; the refill rate sets the sustained rate.
symptoms
- need to rate-limit but allow short bursts
- fixed-window limits cause edge-of-window storms
causes
- naive counter-based rate limiting is too rigid
- leaky bucket smooths but disallows bursts
fixes
- token bucket with capacity = burst size, refill rate = sustained rate
- per-identity buckets in Redis or local cache
- reject with 429 + Retry-After
you might say
- token bucket
- burst-tolerant rate limit