Bulkhead Implementation: Thread Pools, Connection Pools, and Semaphores

Thread Pool Isolation
Each downstream dependency gets its own thread pool. Requests to Service A execute on Pool A; requests to Service B execute on Pool B. When Pool A is exhausted, new A requests are rejected immediately rather than queuing. Implementation requires routing logic to direct requests to the correct pool and handling pool exhaustion gracefully.
Connection Pool Isolation
Database and HTTP connections are limited resources. A shared connection pool of 100 connections can be exhausted by one slow query pattern. Separate pools per database or per query type prevent this. Critical read path gets 60 connections; analytics queries get 20; background jobs get 20. Slow analytics cannot block user requests.
Semaphore Isolation
Lighter weight than thread pools. A semaphore limits concurrent executions without dedicating threads. Set semaphore permits to 20 for Service A calls; when 20 requests are in flight, additional requests fail immediately. Uses less memory than thread pools but provides less isolation since requests still share the main thread pool for execution.
⚠️ Key Trade-off: Thread pool isolation provides strongest isolation but highest overhead (memory per thread, context switching). Semaphores are lightweight but share underlying threads. Choose based on isolation requirements and resource constraints.
Sizing Bulkhead Pools
Size based on expected concurrency and latency. If Service A handles 100 RPS with 50ms latency, you need 100 × 0.05 = 5 concurrent threads minimum. Add buffer for variance: 2-3x gives 10-15 threads. Too small causes unnecessary rejections; too large wastes resources and reduces isolation benefit.
Queue Configuration
Each pool can have a bounded queue for brief bursts. Queue size trades latency for throughput: longer queues absorb spikes but increase wait time. For latency sensitive paths, use small queues (5-10) or no queue (fail immediately when pool exhausted).

💡 Key Takeaways

✓Thread pool isolation: strongest isolation, highest overhead. Each dependency gets dedicated threads.

✓Semaphore isolation: lightweight, limits concurrency without dedicated threads, weaker isolation

✓Sizing formula: RPS × latency × 2-3x buffer. 100 RPS at 50ms needs 10-15 threads minimum.

📌 Interview Tips

1Calculate pool size: 100 RPS × 50ms = 5 concurrent minimum, 2-3x buffer = 10-15 threads

2Compare isolation types: thread pools for critical paths needing strong isolation, semaphores for less critical

3Mention queue sizing: small queues (5-10) for latency sensitive, larger for throughput focused

← Back to Bulkhead Pattern Overview