Handling Scale: WebSockets and Connection Management

Connection Management at Scale
Each tracking device maintains persistent connection. 1 million devices means 1 million concurrent connections. Standard HTTP servers handle maybe 10,000 connections per instance. Need 100+ servers just for connections, before any processing.
WebSocket servers optimize for many idle connections. Event-driven architecture (Node.js, Go) handles 100,000+ connections per server. Use connection pooling and multiplexing where possible. Monitor connection count per server and memory usage.
WebSocket vs Alternatives
WebSocket: Persistent bidirectional connection. Good for high-frequency updates. Requires sticky load balancing or connection-aware routing. Stateful servers complicate scaling.
Server-Sent Events (SSE): Server pushes to client. Simpler than WebSocket. Good when server needs to push but client updates are infrequent. Uses standard HTTP, easier load balancing.
Polling: Client requests at intervals. Stateless, simple scaling. But high overhead for frequent updates. Acceptable for low-frequency tracking like package delivery.
Handling Disconnects
Mobile connections drop frequently: tunnels, poor signal, battery saver. Design for disconnection as normal state. Store last known position with timestamp. Mark stale positions in query results. Reconnect with exponential backoff to avoid thundering herd.
Detect ghost connections: device disconnected but server did not notice. Heartbeat every 30 seconds. No heartbeat for 2 minutes means disconnected. Clean up stale connections to free resources.
💡 Key Insight: Connection count is often the scaling bottleneck, not processing. A million connections with 1 KB each is 1 GB of connection state alone. Plan for connection overhead before worrying about message processing.

💡 Key Takeaways

✓1 million devices need specialized servers; standard HTTP handles 10K connections

✓Event-driven servers (Node.js, Go) handle 100K+ connections per instance

✓WebSocket for high-frequency; SSE for push-only; polling for low-frequency

✓Design for disconnection: last known position with timestamp, stale markers

✓Heartbeat detects ghost connections; exponential backoff prevents thundering herd

📌 Interview Tips

1Calculate connection scaling: 1M devices, 10K connections per server means 100+ servers just for connections

2Compare protocols: WebSocket for ride tracking, polling for package tracking

3Explain ghost detection: no heartbeat for 2 minutes triggers cleanup of connection state

← Back to Real-time Location Tracking Overview