Privacy & Fairness in ML • Fairness Metrics (Demographic Parity, Equalized Odds)Medium⏱️ ~2 min
Demographic Parity vs Equalized Odds: When to Choose Each
Demographic parity and equalized odds represent fundamentally different fairness philosophies. Demographic parity enforces allocation fairness by equalizing selection rates, ignoring whether individuals are actually qualified. Equalized odds enforces procedural fairness by equalizing error rates conditional on qualification. The choice depends on the harm you are preventing and domain constraints.
Demographic parity fits scenarios where you distribute fixed resources and want equal access regardless of historical patterns. Content recommendation systems at Meta and Google use parity style metrics to ensure equal exposure across demographic groups, preventing filter bubbles. The metric requires only predictions, enabling real time monitoring. However, when base rates legitimately differ, enforcing parity forces higher false positive rates in lower scoring groups, reducing accuracy by 2 to 10 percentage points.
Equalized odds fits high stakes risk decisions like fraud detection, credit, and safety where equal error rates matter more than equal selection rates. A fraud model with unequal FPR across groups imposes different costs of false accusations. Microsoft and Amazon enforce equal opportunity (equal TPR only) for hiring and credit to ensure qualified applicants have equal chances. The challenge is label delay: you cannot compute TPR and FPR until outcomes are known, typically 30 to 90 days later. This limits equalized odds to offline evaluation and weekly monitoring.
The mathematical reality is that you cannot simultaneously achieve demographic parity, equalized odds, and calibration when base rates differ across groups. Calibration means predicted scores reflect true risk consistently across groups. Enforcing equalized odds breaks calibration, requiring different thresholds per group. Some jurisdictions prohibit group specific thresholds, making equalized odds legally infeasible despite being technically achievable.
💡 Key Takeaways
•Demographic parity targets allocation fairness for resource distribution (content, interviews, loans). Equalized odds targets procedural fairness for risk decisions (fraud, credit, safety)
•Parity enables real time monitoring because it needs only predictions. Equalized odds requires delayed labels (30 to 90 days), limiting it to weekly batch evaluation
•Enforcing parity with differing base rates inflates false positives by 2 to 10 percentage points. Enforcing equalized odds requires per group thresholds, breaking calibration
•You cannot satisfy demographic parity, equalized odds, and calibration simultaneously when base rates differ. This is a mathematical impossibility, not an engineering limitation
•Some jurisdictions prohibit group specific decision thresholds, making equalized odds legally infeasible despite technical feasibility. Always check regulatory constraints
•Production systems at Google and Microsoft typically use demographic parity for exposure and equal opportunity (relaxed equalized odds) for high stakes decisions with 5 percentage point TPR gap limits
📌 Examples
Meta content recommendation: Uses demographic parity to ensure equal impression rates across age groups, monitored in real time over 100,000 decision sliding windows
Amazon credit scoring: Uses equal opportunity to ensure qualified borrowers have equal approval rates across race groups, recomputed weekly when 90 day default labels arrive
Google hiring classifier: Enforces demographic parity ratio above 0.85 for interview invitations, then equal opportunity for final offers to balance access and merit