Explainable ML for Healthcare

Overview

A healthcare model that cannot explain itself will not be trusted or deployed. This skill makes models from gradient-boosted trees to deep nets interpretable and auditable, covering both why this prediction (local) and how the model behaves (global), plus a fairness audit across patient subgroups. Explainability here is a first-class requirement, not an afterthought.

When to Use This Skill

• Explaining individual predictions to clinicians or reviewers (case-level reasons).
• Global feature-importance and behavior analysis for model debugging.
• Interpreting deep-learning models (text, imaging, sequence) via attribution.
• Auditing fairness/bias across age, sex, race/ethnicity, geography, or payer.
• Producing model cards and documentation for ml-model-validation-regulatory.

Techniques

1. SHAP Shapley-value attributions; TreeExplainer for boosted trees, DeepExplainer/

GradientExplainer for nets, KernelExplainer as a model-agnostic fallback. Global (beeswarm/bar) + local (waterfall/force).

1. LIME local surrogate models for quick, model-agnostic case explanations.
2. Deep-net attribution Integrated Gradients and attention for text/sequence;

Grad-CAM saliency for medical images (pairs with imaging skills).

1. Global behavior partial-dependence and ALE plots (ALE is robust to correlated

features, common in clinical data); permutation importance.

1. Counterfactuals minimal feature change to flip a decision ("what would lower this

risk score").

1. Fairness subgroup performance, calibration-within-group, and disparity metrics;

report, don't hide, gaps.

Example

import shap, xgboost as xgb
model = xgb.XGBClassifier().fit(X_train, y_train)

explainer = shap.TreeExplainer(model)
sv = explainer(X_test)
shap.plots.beeswarm(sv)                      # global drivers
shap.plots.waterfall(sv[0])                  # one patient's explanation

# Deep model attribution with Integrated Gradients (Captum)
from captum.attr import IntegratedGradients
ig = IntegratedGradients(net)
attributions = ig.attribute(input_ids, baselines=baseline, target=1)

# Fairness: performance + calibration by subgroup
from sklearn.metrics import roc_auc_score
for g, idx in groups.items():
    print(g, "AUROC", round(roc_auc_score(y[idx], p[idx]), 3),
             "pos-rate", round(p[idx].mean(), 3))

Cautions

Interpret responsibly: SHAP/LIME show association, not causation; explanations can be unstable under correlated features (prefer ALE over PDP, TreeSHAP over KernelSHAP where possible); and a plausible-looking explanation of a biased model is still a biased model. Validate that explanations are stable and clinically sensible before showing them to users.

Outputs

• global_importance.png + feature_importance.csv.
• local_explanations/ per-case SHAP/LIME/IG artifacts.
• fairness_audit.md subgroup metrics, calibration, disparities, recommendations.
• model_card.md interpretability + fairness section for governance.

Healthcare Context

Targets the clinical trust and equity bar: case-level reasons clinicians can sanity-check, and explicit subgroup fairness because healthcare models can encode disparities. Consumes models from healthcare-predictive-modeling; feeds ml-model-validation-regulatory. Complements the repo's existing shap skill with healthcare framing, deep-net attribution, and fairness.

References

• Lundberg & Lee (2017), A Unified Approach to Interpreting Model Predictions (SHAP).
• Ribeiro et al. (2016), "Why Should I Trust You?" (LIME); Sundararajan et al. (2017), IG.
• Mitchell et al. (2019), Model Cards; Captum https://captum.ai