head-of-line blocking pattern
One slow item at the front of a queue stalls everything behind it, even if those items would be fast on their own. The queue's slowest item becomes everyone else's latency.
One slow item at the front of a queue stalls everything behind it, even if those items would be fast on their own. The queue's slowest item becomes everyone else's latency.
symptoms
- p99 dominated by a small number of pathological items
- queue depth growing while throughput stays low
- fast operations starved behind slow ones
causes
- single FIFO queue without parallelism per item type
- shared connection pool with long-running queries
- HTTP/1.1 keep-alive serializing requests
fixes
- separate queues for fast/slow lanes
- timeouts on individual items
- concurrent processing (HTTP/2, parallel workers)
- fairness scheduling
you might say
- one slow query is blocking the whole pool
- head of line blocking on the queue