Database Design • Time-Series Databases (InfluxDB, TimescaleDB)Hard⏱️ ~3 min
Series Cardinality: The Hidden Killer of Time Series Database Performance
Series cardinality refers to the number of unique tag combinations in your time series data. Each unique combination of tag key value pairs creates a separate series that requires its own in memory index entry. This matters because unbounded cardinality is the number one cause of production Time Series Database (TSDB) failures, manifesting as out of memory kills, ingestion timeouts, and query degradation.
The explosion happens faster than expected. Consider a simple setup tracking HTTP requests with tags for endpoint, method, status code, and region. With 100 endpoints, 5 methods, 10 status codes, and 4 regions, you already have 100 × 5 × 10 × 4 = 20,000 unique series. Now imagine a developer adds user ID as a tag to debug an issue. With 1 million active users, cardinality jumps to 20 billion series. Each series maintains metadata in memory including the tag dictionary, last write timestamp, and index pointers. At even 1 kilobyte per series (a conservative estimate), 20 billion series requires 20 terabytes of RAM just for metadata.
Datadog enforces strict per metric tag limits and uses top k rollups specifically to prevent this failure mode. Their platform serves billions of metrics but caps the unique tag combinations per metric and automatically aggregates long tail dimensions into an "other" bucket. Netflix Atlas similarly uses client side aggregation to reduce cardinality before data even reaches the central system. These are not optional niceties but mandatory guardrails.
Mitigations form a defense in depth strategy. First, establish tag governance with whitelisted tag keys and banned high cardinality fields like UUIDs, request IDs, or email addresses. Second, implement per tenant and per metric cardinality quotas with circuit breakers that reject writes exceeding limits. Third, use approximate top k aggregation to preserve the most important dimensions while rolling up rare combinations. Fourth, monitor series growth rate and set alerts when cardinality increases faster than expected. The cost of prevention is inconvenience during development; the cost of failure is a production outage.
💡 Key Takeaways
•Series cardinality equals the product of all unique tag values, so adding a user ID tag with 1 million users to a metric with 20,000 existing series explodes to 20 billion series requiring terabytes of RAM
•Datadog and Netflix enforce strict tag governance with whitelisted keys, per metric cardinality limits, and top k rollups that aggregate long tail dimensions to prevent memory explosions
•At 100,000 plus unique series, tagset based TSDBs encountered timeouts and out of memory crashes in benchmarks without careful batch sizing and backpressure, while relational systems with predictable indexing remained stable
•Memory costs 100 to 1000 times more than disk per gigabyte, making in memory index growth from high cardinality the first bottleneck before disk capacity or I/O limits
•High cardinality symptoms include out of memory kills, ingestion timeouts exceeding service level objectives, and compaction backlogs as the system struggles to merge thousands of small overlapping files
📌 Examples
A monitoring team added container ID as a tag for debugging. With 10,000 containers restarting hourly, they created 240,000 new series per day. Within a week the TSDB crashed with out of memory errors and required emergency tag removal plus data deletion.
Uber M3 implements per tenant cardinality budgets with circuit breakers. When a tenant exceeds their series quota, new tag combinations get rejected with backpressure while existing series continue writing to maintain partial availability.
A streaming platform accidentally logged video session IDs as tags. With 100 million daily sessions, this created billions of series. They recovered by migrating session IDs to fields (unindexed) and using top 1000 aggregation for dimensions like device type.