Real-Time Subscriptions and Fan-Out Cost at Scale

Document databases like Firestore offer real-time change streams that push updates to subscribed clients instantly. When a document changes, all active listeners receive the update without polling. This pattern powers collaborative apps, live dashboards, and chat systems with minimal client complexity. The trade-off is fan-out cost: each write triggers notifications to all subscribers, and in operation-billed systems like Firestore, cost scales linearly with subscriber count and update rate.

Consider a chat room with 100,000 active subscribers and 1 message per second. Each message write triggers approximately 100,000 read events (one per subscriber receiving the update). At Firestore multi-region pricing of roughly $0.06 per 100,000 document reads, that is $0.06 per second or $216 per hour. Real-time convenience is excellent for small to medium scale, but at large scale this model can exceed database compute costs by orders of magnitude. The cost is predictable but grows linearly: doubling subscribers or message rate doubles the bill.

Mitigation strategies include narrowing listener scope and batching. Instead of subscribing to an entire chat room, subscribe to per-user channels or paginated subsets (only the latest 50 messages). Server-side batching (where supported) coalesces multiple updates into fewer notifications. For presence indicators ("user is typing") or high-frequency state changes, use short Time To Live (TTL) documents that expire automatically, keeping churn cheap and avoiding unbounded history growth.

Back-pressure is another failure mode. If update rate outpaces client consumption, listeners disconnect or fall behind, triggering retry storms. Firestore and similar systems enforce per-client quotas and rate limits to prevent cascading failures. At companies like Google, real-time subscriptions power Google Docs collaboration where each keystroke updates shared state to dozens of collaborators. They mitigate cost by scoping listeners to active document sections and using operational transforms to reduce redundant updates. The pattern works beautifully for collaborative tools and dashboards but requires careful cost modeling and scoping for high-traffic consumer apps.

💡 Key Takeaways

•Real-time subscriptions trigger fan-out: 1 write to document with 100K listeners generates 100K read events, at $0.06 per 100K reads that is $0.06 per write, $216/hour at 1 write/sec

•Cost scales linearly with subscribers and update rate: doubling chat room size from 50K to 100K subscribers doubles notification cost from $108/hour to $216/hour, critical for budgeting

•Narrow listener scope to reduce fan-out: subscribe to per-user channels or latest 50 messages instead of entire room, reducing active listeners from 100K to 5K cuts cost by 20x

•High-frequency updates like "typing indicators" should use short TTL documents (expire after 5 seconds) to avoid unbounded history and keep per-update cost low with automatic cleanup

•Back-pressure and quota enforcement: if server pushes updates faster than client consumes, listeners disconnect and retry, creating retry storms, systems enforce per-client rate limits to prevent cascading failures

•Production pattern at Google Docs: scope listeners to active document sections (current page, not entire 100 page doc), use operational transforms to coalesce redundant updates, keeps real-time UX with manageable cost

📌 Examples

Firestore listener cost: chat app with 100,000 users subscribed to room, 1 message/sec generates 100,000 reads/sec, 8.64 billion reads/day, ~$5,184/month in read costs alone at multi-region pricing

Mitigation: limit listeners to recent 50 messages via query .orderBy('timestamp').limit(50), reduces active notifications from 100K to typical 5K concurrent viewers, cost drops to $500/month

Google Docs collaboration: each user subscribes only to visible paragraphs (not entire doc), keystroke updates push to ~10 active collaborators not 1000 doc viewers, fan-out bounded to active editors

Typing indicator pattern: write { userId: 'u123', typing: true, ttl: Date.now() + 5000 } with TTL expiry, auto-deletes after 5 seconds, prevents unbounded typing history accumulation

← Back to Document Databases (MongoDB, Firestore) Overview