Failure Modes and Edge Cases

When Approximation Breaks Down: AQP has specific failure modes that can produce systematically wrong results, not just noisy estimates. Understanding these is critical for system design interviews and production deployments. Rare Events and High Selectivity: The most common failure is querying for rare events on sampled data. If you maintain a 1% sample of 1 billion rows, an event that occurs 10,000 times in the full data appears only about 100 times in the sample. Your confidence intervals explode. Worse, something occurring 500 times globally might be completely absent from your sample, giving you zero instead of 500. This is not just statistical noise. It is systematic undercounting. Imagine debugging a production issue where a specific error code appeared 200 times in the last hour across billions of requests. Your 0.1% sample likely has zero instances. The approximate system tells you the problem does not exist when it actually does. Skew and Distribution Bias: Uniform sampling assumes uniform distribution, but real world data is heavy tailed. A handful of very large customers might generate 50% of traffic. If your sampling is purely random, you might undersample these critical accounts in one draw and oversample in another, leading to high variance in metrics like "average revenue per customer." Stratified sampling helps by oversampling important segments, but it requires knowing the stratification keys in advance. If your query filters on a dimension you did not stratify by, the bias remains. For example, you stratified by country but the query filters by device type. Your sample distribution across device types does not match the population. Sketch Merging and Multi Stage Aggregation: While sketches are mathematically mergeable, errors can compound in complex queries. If you merge sketches across 1,000 partitions, each with its own ±1% error, the combined error is not simply ±1%. It depends on overlap patterns and can grow to ±3% or more in pathological cases. Multi stage aggregations amplify this. Imagine computing distinct users per country, then rolling up to distinct users per continent. The continent level sketch merges country level sketches, each already approximate. If a user appears in multiple countries (for example, travelers), the sketch might double count or undercount depending on merge order and hash collisions. Staleness and Drift: If samples or sketches refresh only nightly, but data distribution shifts during the day, your error bounds become invalid. Imagine a flash sale that triples traffic from a specific country. Your sketch from yesterday dramatically underestimates today's distinct users from that country because the sketch size was tuned for normal traffic. In streaming systems, delayed or out of order events create temporal inconsistencies. If an event arrives 2 hours late but your sketch already rolled over to a new time window, that event is either lost or counted in the wrong bucket. At high throughput (1 million events per second), even 0.1% late arrivals mean 1,000 miscounted events per second. Human and Operational Failures: The hardest failures are not mathematical but human. If users do not understand that "10.2M ± 1.5%" is a probabilistic statement, they might compare it directly against an exact metric from another system and report discrepancies as bugs. If your User Interface (UI) does not clearly label results as approximate, people make decisions assuming perfect accuracy. Monitoring is critical. Track what fraction of queries use AQP, actual observed error rates against periodic ground truth, and divergence over time. Implement circuit breakers: if error estimates exceed a threshold (for example, ±10%), automatically fall back to exact queries even if slower. Without this, you risk making business decisions on systematically biased data.