Production Scale: The Full Data Lifecycle

The Reality of Scale: At companies like Uber or Airbnb handling billions of events daily, masking and anonymization aren't single step operations. They're enforced across the entire data lifecycle: ingestion, storage, batch processing, real time serving, and external sharing. Each stage has different latency requirements and risk profiles. Stage 1: Ingestion and Early Masking: Events land at the edge at 100k per second through Kafka or Kinesis. Certain fields are never stored raw. Email addresses get hashed immediately using SHA-256 with application specific salt, producing consistent tokens across all downstream systems. IP addresses are truncated to /24 CIDR blocks (city level) before hitting storage. GPS coordinates are rounded to 3 decimal places (roughly 100 meter precision) instead of the raw 8 decimal place data from mobile devices. This early transformation is critical: it means your data lake and warehouse never contain raw PII, dramatically reducing the blast radius if storage is compromised. The latency cost is minimal because hashing and truncation are CPU bound operations taking microseconds per field. Stage 2: Role Based Query Time Masking: In the warehouse serving 1k to 5k queries per second, different roles see different views of the same data. Analysts querying user behavior see tokenized user_id but not emails or phone numbers. Customer support has access to last 4 digits of phone numbers for verification. ML training jobs get heavily aggregated features: not individual ages but age buckets (25 to 34), not exact locations but metropolitan regions. This is implemented through view layers or query rewriting. When a user with role