Privacy & Fairness in ML • Model Interpretability (SHAP, LIME)Medium⏱️ ~3 min
What are SHAP and LIME for Model Interpretability?
SHapley Additive exPlanations (SHAP) and Local Interpretable Model agnostic Explanations (LIME) are two dominant frameworks for explaining individual predictions from black box models. Both are model agnostic and focus on local explanations: why did this model produce this specific output for this particular input.
SHAP is grounded in Shapley values from cooperative game theory. It assigns each feature a contribution value such that all contributions sum to the difference between the prediction and a baseline. For a credit score model predicting 720 points with a baseline of 680, SHAP might attribute +25 points to income, +10 to payment history, and +5 to credit age. SHAP satisfies critical axioms like local accuracy (explanations sum to the actual prediction) and consistency (if a feature helps more, its attribution never decreases). Amazon SageMaker Clarify uses SHAP to generate bias reports and feature attributions for production models.
LIME takes a different approach focused on local fidelity. It perturbs the input hundreds or thousands of times (changing feature values randomly), queries the black box model on these perturbed samples, then fits a simple surrogate model like linear regression in that neighborhood. The surrogate's coefficients become the explanation. If you perturb a loan application 1,000 times and fit a linear model, you might find that income has coefficient 0.8 and debt ratio has coefficient negative 0.6. LIME is faster to prototype but can be unstable because results depend on random perturbations, kernel width, and neighborhood size.
In production, the choice matters for both engineering and governance. SHAP offers stability and theoretical guarantees but can be computationally expensive on neural networks. LIME is intuitive and fast for local debugging but too variable for regulatory contexts requiring reproducible explanations.
💡 Key Takeaways
•SHAP assigns feature contributions that sum to the prediction minus baseline, using Shapley values from game theory with guarantees like local accuracy and consistency.
•LIME perturbs inputs hundreds to thousands of times, queries the model, then fits a simple surrogate (often linear) to approximate the local decision surface.
•For tree ensembles with 100 trees and depth 6, SHAP computes attributions in 2 to 5 milliseconds per instance on CPU, enabling synchronous serving at scale.
•LIME typically requires 500 to 2,000 model evaluations per instance, taking 0.5 to 2 seconds, making it suitable only for offline analysis or asynchronous APIs.
•SHAP offers stability and reproducibility critical for regulatory compliance (adverse action notices, audit trails), while LIME excels in fast iteration during model debugging.
•Both depend on careful configuration: SHAP requires a representative background dataset (500 to 2,000 stratified samples), LIME requires realistic perturbation strategies to avoid generating invalid inputs.
📌 Examples
Amazon SageMaker Clarify uses SHAP based attributions for bias detection and explainability reports, computing feature contributions for credit risk and fraud models at scale.
Microsoft Azure Machine Learning provides a unified interpretability dashboard combining SHAP local attributions with global feature importance and fairness metrics across model types.
Google Cloud AI uses SHAP for tabular models and gradient based methods for neural networks, integrated into Vertex AI pipelines with versioned explanation artifacts.
A credit risk platform serving 5,000 predictions per second dedicates 20 to 50 CPU cores to compute top 3 to 5 SHAP attributions within 15 milliseconds for customer support and adverse action notices.