thread pool pattern
Instead of one-thread-per-task, keep N pre-created threads that pull tasks from a queue. Bounds memory use, avoids thread-creation cost. The size N is a critical tuning knob: too small starves throughput, too large starves the rest of the system or causes context-switch thrash.
Instead of one-thread-per-task, keep N pre-created threads that pull tasks from a queue. Bounds memory use, avoids thread-creation cost. The size N is a critical tuning knob: too small starves throughput, too large starves the rest of the system or causes context-switch thrash.
symptoms
- unbounded threads on traffic spikes
- OOM from thread overhead
- context switching dominating CPU
causes
- one-thread-per-task model
- no concurrency cap
fixes
- pool size = a function of CPU cores or downstream concurrency limits
- queue with bounded length
- reject when full (load shed)
you might say
- thread pool
- worker pool
- goroutine pool