SystemOverflow
Learn→Privacy & Fairness in ML→Model Interpretability (SHAP, LIME)→3 of 5
Privacy & Fairness in ML • Model Interpretability (SHAP, LIME)Medium⏱️ ~3 min

SHAP vs LIME vs Gradient Methods: Choosing the Right Technique

SHAP Strengths and Weaknesses

Strengths: Theoretically sound (Shapley values). Consistent: identical features get identical attributions. Additive: contributions sum to prediction minus baseline. TreeSHAP is exact and fast (10-50ms) for tree models. Weaknesses: Computationally expensive for non-tree models. KernelSHAP approximation can be unstable. Assumes feature independence, which rarely holds.

LIME Strengths and Weaknesses

Strengths: Model-agnostic. Faster than SHAP (10-50ms). Intuitive output (linear coefficients). Works for tabular, text, and images. Weaknesses: Inconsistent: similar inputs may get different feature rankings. Depends on perturbation strategy. No theoretical correctness guarantee. Can miss non-linear boundaries.

Gradient-Based Methods

For neural networks, compute how each input affects output via partial derivatives. Integrated Gradients: Accumulates gradients along path from baseline to input. Satisfies completeness. Fast (single forward/backward pass, 5-20ms). Saliency maps: Gradient magnitude. Very fast but noisy. Best for deep learning where SHAP/LIME are too slow.

Decision Framework

Tree models: TreeSHAP (fast, exact). Regulatory requirements: SHAP for theoretical soundness. Real-time latency: LIME or gradients. Neural networks: Integrated Gradients. Need consistency: Avoid LIME. Need speed: LIME or distilled models.

💡 Key Insight: No method is universally best. SHAP for correctness, LIME for speed, gradients for neural networks. Match method to model type and requirements.
💡 Key Takeaways
✓SHAP: theoretically sound, consistent, additive; but slow and assumes independence
✓TreeSHAP is exact and fast (10-50ms) for tree-based models
✓LIME: model-agnostic and fast; but inconsistent and depends on perturbation
✓Gradient methods: fast (5-20ms) for neural networks, Integrated Gradients preferred
✓Decision: TreeSHAP for trees, SHAP for regulation, LIME for speed, gradients for NNs
📌 Interview Tips
1Provide decision matrix: model type to explanation method
2Note LIME consistency problem: similar inputs may get different rankings
← Back to Model Interpretability (SHAP, LIME) Overview