Communication Costs: Shared Memory vs IPC

Shared Memory Communication
Threads communicate by reading and writing shared heap memory. One thread writes a value; another reads it. The cost is essentially memory access: ~100 nanoseconds for a cache miss, or 1-10 nanoseconds if data is in CPU cache.
However, shared memory requires synchronization. Without locks or atomic operations, two threads writing the same location create a race condition. An uncontended mutex costs 25-50 nanoseconds; a contended mutex can cost microseconds as threads wait.
Inter Process Communication
Processes cannot access each others memory directly. They use IPC (Inter Process Communication) mechanisms involving the kernel. Every IPC operation requires at least two context switches: one to enter the kernel, one to return.
Pipes and sockets copy data from sender to kernel buffer, then to receiver. Sending 1KB through a local socket takes 2-10 microseconds, roughly 100x slower than shared memory.
Shared memory regions let processes map the same physical memory into their address spaces. After setup, access is as fast as regular memory, but cross process synchronization primitives are needed.
Serialization Overhead
IPC often requires serialization: converting in memory structures to bytes. A 1KB message takes 1-5 microseconds with binary formats, or 10-50 microseconds with JSON.
The Performance Gap
For one million messages: threads with a lock free queue complete in ~100ms. Processes with sockets take ~10 seconds, a 100x slowdown.
⚠️ Key Trade-off: Shared memory gives nanosecond latency but requires careful synchronization. IPC gives isolation but costs microseconds per operation.

💡 Key Takeaways

✓Shared memory communication costs ~100 nanoseconds per access; socket based IPC costs 2-10 microseconds, roughly 100x slower

✓Uncontended mutex operations add 25-50 nanoseconds; contended mutexes can stall threads for microseconds

✓IPC requires kernel involvement with at least two context switches per operation, adding unavoidable overhead

✓Serialization adds 1-50 microseconds depending on format; binary formats are 10x faster than JSON for equivalent data

✓For one million messages, threads complete in ~100ms while processes with sockets take ~10 seconds

📌 Interview Tips

1When designing high throughput systems, calculate IPC overhead: 10 microseconds per message means 100,000 messages per second maximum, regardless of CPU speed

2If asked about process vs thread for a data pipeline, explain that shared memory avoids serialization entirely while IPC requires converting objects to bytes and back

3Mention memory mapped files as a middle ground: processes get isolation, but communication approaches shared memory speed after the initial mapping setup

← Back to Processes vs Threads Overview