SystemOverflow
Learn→Privacy & Fairness in ML→Model Interpretability (SHAP, LIME)→5 of 5
Privacy & Fairness in ML • Model Interpretability (SHAP, LIME)Hard⏱️ ~3 min

Implementation Patterns: From Prototyping to Production Governance

Prototyping Phase

Start with library defaults. Use shap.Explainer(model) or LIME with standard settings. Generate explanations for a sample. Visualize to validate: if random noise features rank highly, debug before proceeding. Goal: confirm the approach works for your model and data. Typical time: 1-2 days.

Production Integration

Build explanation service separate from prediction. API: given prediction ID, return cached explanation or compute on-demand. Storage: JSON with feature names, values, importances. Indexed by prediction ID and timestamp. Set 90-day retention for regulated domains. Add circuit breakers: if explanation fails, log error but do not fail the prediction request.

Governance and Audit

Model cards: Document explanation method, limitations, and failure modes. Versioning: Store model version with each explanation since they change with model updates. Audit log: Record who accessed which explanations. Human review: Periodically sample explanations for domain expert validation.

User-Facing Explanations

Raw SHAP values are not user-friendly. Translate: "income: -0.3" becomes "Your income of ,000 is below the typical approved range." Use templates with thresholds. Top-3 features only. Users prefer contrastive explanations: "if income were K instead of K, approval would increase 15%."

💡 Key Insight: Technical explanations are for engineers and auditors. User explanations need translation into actionable, natural language.
💡 Key Takeaways
✓Start with library defaults, validate top features make sense before production
✓Separate explanation service: cache results, handle failures gracefully
✓Store model version with each explanation for audit when models update
✓90-day retention for regulated domains with access logging
✓Translate SHAP to natural language: contrastive explanations (if X were Y, result changes Z%)
📌 Interview Tips
1User preference: contrastive format ("if X were Y, probability increases Z%")
2Governance: model cards document method, limitations, failure modes
← Back to Model Interpretability (SHAP, LIME) Overview
Implementation Patterns: From Prototyping to Production Governance | Model Interpretability (SHAP, LIME) - System Overflow