Learn→Privacy & Fairness in ML→Fairness Metrics (Demographic Parity, Equalized Odds)→2 of 6

Privacy & Fairness in ML • Fairness Metrics (Demographic Parity, Equalized Odds)Easy⏱️ ~2 min

What is Equalized Odds?

Definition
Equalized Odds requires that the true positive rate (TPR) and false positive rate (FPR) be equal across demographic groups. The model should make errors at the same rate for everyone, conditioned on actual outcome.
The Mathematical Definition
Equalized odds has two requirements. First: P(Ŷ=1|Y=1,A=0) = P(Ŷ=1|Y=1,A=1). Among people who truly qualify (Y=1), approval rate should be equal. Second: P(Ŷ=1|Y=0,A=0) = P(Ŷ=1|Y=0,A=1). Among those who do not qualify, false approval rate should also be equal. If 90% of qualified women get approved, 90% of qualified men must too.
Why This Matters
Unlike demographic parity, equalized odds respects ground truth. If Group A has more qualified applicants, they get more approvals, but the model treats qualified and unqualified individuals from both groups equally. A medical diagnostic should catch 95% of cancer cases in both men and women. A fraud system should flag 5% of legitimate transactions falsely in both groups.
The Core Limitation
Equalized odds assumes ground truth labels are correct. If historical labels were biased (qualified women labeled unqualified), equalized odds perpetuates bias. It also requires ground truth for evaluation, which may not exist. For loan default, you only know if someone defaults after giving them the loan. Without outcomes for rejected applicants, measuring equalized odds is impossible.
⚠️ Key Trade-off: Equalized odds trusts the labels. If labels are biased, enforcing this metric locks in that bias. Audit labels before relying on it.

💡 Key Takeaways

✓Requires equal true positive rate AND equal false positive rate across groups

✓Respects ground truth: groups with more qualified members get more approvals

✓Error rates should be consistent: 95% cancer detection for both groups, not 95% vs 80%

✓Assumes labels are correct: biased labels mean equalized odds perpetuates bias

✓Requires ground truth for evaluation, which may not exist for rejected applicants

📌 Interview Tips

1Explain both conditions: equal TPR among qualified, equal FPR among unqualified

2Mention label trust assumption: biased labels mean the metric fails to detect true issues

← Back to Fairness Metrics (Demographic Parity, Equalized Odds) Overview