Learn→Privacy & Fairness in ML→Fairness Metrics (Demographic Parity, Equalized Odds)→3 of 6

Privacy & Fairness in ML • Fairness Metrics (Demographic Parity, Equalized Odds)Medium⏱️ ~2 min

Demographic Parity vs Equalized Odds: When to Choose Each

The Fundamental Tension
Demographic parity and equalized odds cannot both be satisfied simultaneously (except in trivial cases). This is mathematically proven: if base rates differ between groups, achieving one metric necessarily violates the other. If 60% of Group A qualifies and 40% of Group B qualifies, equal approval rates means different error rates, and vice versa. You must choose.
When to Choose Demographic Parity
Historical bias dominates: If past data reflects systemic discrimination rather than true qualification differences. Hiring from eras when groups were excluded does not reflect ability. Base rates are unreliable: If observed differences stem from biased measurement or unequal access rather than true differences. Representation matters: When diverse representation itself creates value (jury selection, political roles), equal rates may be the goal regardless of qualifications.
When to Choose Equalized Odds
Labels are trustworthy: If ground truth labels are accurate (medical diagnoses confirmed by tests, fraud confirmed by investigation), equalized odds ensures the model does not harm one group. Individual fairness matters: Equal error rates mean equally qualified people have equal chances. Base rate differences are legitimate: If qualification differences reflect genuine factors (age correlates with health conditions), forcing equal rates would be inappropriate.
Decision Framework
Ask: Do you trust the labels? Biased labels make equalized odds perpetuate bias. Do you trust the base rates? If observed differences reflect true differences, demographic parity penalizes qualified individuals. Production systems often use relaxed versions tolerating 10-20% deviation.
💡 Key Insight: The choice between metrics is philosophical, not technical. What does fair mean in your context? Different stakeholders may reasonably disagree.

💡 Key Takeaways

✓Both metrics cannot be satisfied simultaneously when base rates differ (impossibility theorem)

✓Choose demographic parity when: historical bias, unreliable base rates, representation value

✓Choose equalized odds when: trustworthy labels, individual fairness, legitimate base rate differences

✓Two key questions: Do you trust the labels? Do you trust the base rates?

✓Production often uses relaxed versions tolerating 10-20% deviation from exact equality

📌 Interview Tips

1Frame choice as philosophical: what does fair mean in your context?

2Mention impossibility theorem proves both cannot be satisfied simultaneously

← Back to Fairness Metrics (Demographic Parity, Equalized Odds) Overview