Learn→A/B Testing & Experimentation→Experiment Design (Randomization, Stratification, Power Analysis)→6 of 6

A/B Testing & Experimentation • Experiment Design (Randomization, Stratification, Power Analysis)Hard⏱️ ~3 min

How Do Geo and Switchback Designs Handle Interference?

Definition
Geo-cluster and switchback experiments address interference where user-level randomization creates contamination. They randomize at region or time-period level instead.
The Interference Problem
In a two-sided marketplace, treating some buyers but not others in the same city affects supply availability for all. If treatment buyers get faster matching, they consume driver supply that control buyers would have used. User-level effects spill over, biasing your estimates.
Social networks have similar issues: if treatment users share more, control users see more content. The control experience is contaminated by treatment effects.
Geo-Cluster Design
Randomize entire cities or regions. Chicago gets treatment, Detroit gets control. All users in a region have same experience, eliminating within-region spillover. Trade-off: you need many regions for statistical power. With 50 cities, you have 25 units per arm - high variance.
💡 Key Insight: Synthetic control methods compare treatment regions to weighted combinations of control regions matched on pre-experiment trends. This reduces variance when region count is limited.
Switchback Design
Alternate treatment and control over time periods (hours, days) within each region. 9-10am treatment, 10-11am control, etc. This multiplies effective sample by number of time periods. Trade-off: carryover effects between periods can contaminate results.

💡 Key Takeaways

✓User-level randomization fails when treatment effects spill over to control users (marketplace, social)

✓Geo-cluster randomizes entire regions but needs many regions (50+) for statistical power

✓Synthetic control matches treatment regions to weighted control combinations for variance reduction

✓Switchback alternates treatment/control over time but risks carryover effects between periods

📌 Interview Tips

1When asked about marketplace experiments: explain supply-side spillover where treatment consumes shared resources

2For geo design: mention 50+ regions needed for power, synthetic control for limited regions

← Back to Experiment Design (Randomization, Stratification, Power Analysis) Overview