Three-Tier Guardrail Framework

Core Concept
The three-tier framework organizes guardrails by response time: immediate (real-time), session-level (same day), and long-term (weeks). Each tier has different detection speed vs accuracy trade-offs.
Tier 1: Immediate Guardrails
Detect within minutes to hours. Error rates, crash rates, latency p99. These catch catastrophic failures fast. Threshold example: >1% error rate increase triggers automatic rollback. Measured via real-time streaming, checked every 5-15 minutes.
Tier 2: Session-Level Guardrails
Detect within hours to 1 day. Session completion rate, add-to-cart rate, bounce rate. These catch UX degradation before it compounds. Threshold example: >5% session bounce increase blocks experiment. Requires waiting for sessions to complete before measurement.
Tier 3: Long-Term Guardrails
Detect over days to weeks. 7-day retention, repeat purchase rate, NPS. These catch delayed harm that session metrics miss. Threshold example: >2% retention drop stops experiment. Requires longer runtime but catches the most important issues.
💡 Key Insight: Tier 1 guardrails auto-rollback without human review. Tier 2 pauses for investigation. Tier 3 informs ship/no-ship decision but experiment continues while measuring.
Action Matrix
Each tier has a response: Tier 1 = auto-rollback (immediate), Tier 2 = pause + alert (same day), Tier 3 = decision input (end of experiment). This tiered approach balances speed with thoroughness.

💡 Key Takeaways

✓Tier 1 (minutes): error rates, crashes, latency - auto-rollback on breach

✓Tier 2 (hours): session completion, bounce rate - pause for investigation

✓Tier 3 (days/weeks): retention, repeat purchase - informs ship decision

✓Faster tiers catch catastrophic failures; slower tiers catch subtle harm

📌 Interview Tips

1When explaining tiers: describe detection speed trade-off (fast = crude, slow = accurate)

2For action matrix: Tier 1 auto-rollback, Tier 2 pause+alert, Tier 3 decision input

← Back to Guardrail Metrics Overview