Learn→Privacy & Fairness in ML→Fairness Metrics (Demographic Parity, Equalized Odds)→6 of 6

Privacy & Fairness in ML • Fairness Metrics (Demographic Parity, Equalized Odds)Hard⏱️ ~3 min

Post Processing Threshold Optimization for Fairness

Core Concept
Post-processing threshold optimization adjusts decision thresholds per group to achieve fairness without retraining. Instead of 0.5 cutoff for everyone, use 0.45 for Group A and 0.55 for Group B.
How It Works
Train your model normally, outputting probability scores. For each group, find the threshold achieving your fairness constraint. For demographic parity: thresholds such that positive rate is equal. For equalized odds: thresholds such that TPR and FPR are equal. This is an optimization problem: search threshold pairs to minimize fairness violation while maximizing accuracy. Grid search works for two groups; constrained optimization for more.
When to Use Post-Processing
Cannot retrain: Model training is expensive or you lack pipeline access. Regulatory compliance: Need quick fairness demonstration without long retraining. Interpretability: Threshold adjustments are easy to explain. However, post-processing is a patch. The underlying model still learned biased patterns. If features change, thresholds need recalibration.
The Accuracy Cost
Adjusting thresholds trades accuracy for fairness. If Group A threshold drops from 0.5 to 0.4, more lower-score members get approved, some as false positives. Typical accuracy loss: 2-5% for demographic parity, 3-8% for equalized odds. Cost depends on base rate differences: similar rates need minimal adjustment, large differences (60% vs 30%) cause larger hits.
⚠️ Key Trade-off: Post-processing is fast and interpretable but treats symptoms. For long-term fairness, combine with in-processing (training constraints) or pre-processing (debiasing data).

💡 Key Takeaways

✓Adjusts decision thresholds per group without retraining: 0.45 for Group A, 0.55 for Group B

✓Use when cannot retrain, need quick compliance, or need interpretable adjustments

✓Optimization: search threshold pairs to minimize fairness violation, maximize accuracy

✓Typical accuracy cost: 2-5% for demographic parity, 3-8% for equalized odds

✓Treats symptoms not causes: underlying model still has biased patterns

📌 Interview Tips

1Explain interpretability: threshold adjustments are easy to explain to stakeholders

2Accuracy cost depends on base rate differences: similar rates need minimal adjustment

← Back to Fairness Metrics (Demographic Parity, Equalized Odds) Overview