GC Throughput vs Latency Trade offs and Memory Overhead

Throughput vs Latency

Throughput collectors maximize application CPU time. They use parallel threads during collection and may stop the world for longer. Acceptable for batch jobs where total completion time matters more than individual response times.

Latency focused collectors minimize pause times. They do more work concurrently with application threads. Overhead is higher but pauses are shorter. Essential for interactive services where P99 latency matters.

Pause Time Characteristics

Parallel collectors: Stop the world for entire collection. Pause scales with heap size. 10 GB heap might pause for 100 to 500 milliseconds. Good throughput, poor latency.

Concurrent collectors: Most work concurrent with application. Short stop the world phases for root scanning and reference updates. Pauses typically 10 to 50 milliseconds regardless of heap size.

Pauseless collectors: ZGC and Shenandoah achieve sub-millisecond pauses even for terabyte heaps. Pauses are O(1) relative to heap size. Higher CPU overhead but predictable latency.

Memory Overhead

Concurrent GC requires more memory. The application continues allocating during collection. The collector must finish before heap exhausts. Rule of thumb: concurrent collectors need 2x the live data size as heap. More headroom reduces collection frequency and failure risk.

Compact collectors use less memory than non-compacting ones. Fragmentation wastes space with scattered free blocks too small to satisfy allocations. Compaction eliminates fragmentation but requires moving objects and updating references.