What are Real-Time OLAP Engines?

Real-time Online Analytical Processing (OLAP) engines are distributed, columnar, Massively Parallel Processing (MPP) systems built to serve complex aggregations with tight Service Level Agreements (SLAs), often sub-second p95 latency, on continuously arriving data. Unlike traditional data warehouses that update hourly or daily, these engines make data queryable within seconds of event creation. Think LinkedIn showing "Who viewed your profile" within seconds, or Uber calculating pricing metrics in near real time. The architecture follows a streaming pattern: event streams flow into a streaming transform layer that denormalizes data and precomputes dimensional keys, then into OLAP storage segments organized by time partitions. The query layer executes vectorized scans with predicate pushdown and Single Instruction Multiple Data (SIMD) friendly column encoding. LinkedIn runs 100+ use cases at petabyte scale with tens of thousands of queries per second, maintaining sub-100 ms p95 latency for filtered aggregations. Uber achieves sub-200 ms p95 while scanning tens of millions of rows with seconds-level freshness. The key innovation is combining fast filters (dictionary encoding, bitmap indexes, range indexes, pre-aggregation trees) with columnar compression to drastically reduce Input/Output (IO). Data is modeled in denormalized, wide fact tables keyed by event time, avoiding expensive joins at query time. Tiered storage keeps hot segments (last 7 to 30 days) on Solid State Drives (SSDs) for instant access while moving warm and cold data to cheaper object storage, increasing query latency from milliseconds to low seconds but cutting storage costs by 50 to 80%. Operationally, "real-time" means end to end freshness of seconds to minutes from event production to queryability, not microsecond per-row latency. Engines implement at-least-once ingestion with idempotent upserts for correctness under retries, segment level replication for availability, and multi-tenancy controls (quotas, query timeouts, segment assignment) to isolate workloads and sustain SLAs at high concurrency.

LinkedIn "Who viewed your profile": Streams profile view events, enriches with viewer metadata, stores in time partitioned segments with bitmap indexes on dimensions (industry, company, location), serves sub-100 ms queries for "views in last 7 days grouped by industry" at thousands of queries per second

Uber marketplace metrics: Ingests trip events at millions per day with seconds-level freshness, precomputes city and hour level aggregations, uses GPU acceleration to scan tens of millions of rows and return pricing signals in sub-200 ms p95

Airbnb experimentation analytics: Reduces experiment metric latency from hours to minutes by streaming booking events into OLAP with minute-level freshness, serves 100s of concurrent analysts with p95 under 1 to 2 seconds using precomputed metrics and result caching