API Reference / explain /

local.py

Local, model-agnostic explanations: LIME surrogates and counterfactuals.

Functions

Func

lime_explain

Line 39
lime_explain(estimator: Any, x: np.ndarray, background: np.ndarray, class_index: Optional[int]=None, n_samples: int=2000, kernel_width: Optional[float]=None, n_features: Optional[int]=None, feature_names: Optional[List[str]]=None, random_state: Optional[int]=None) -> Explanation

Explain one prediction by fitting a local linear model around it.

LIME's premise is that a model too complicated to describe globally is often nearly linear in a small neighbourhood. It perturbs the sample, asks the model what it thinks of each perturbation, weights those by how close they landed, and fits a weighted linear model. The coefficients are the explanation — and because only predict is ever called, this works on any model, including ones TreeExplainer cannot touch.

Overview

  1. Draw perturbations by sampling each feature from the background's
distribution.
  1. Query the model on every perturbation.
  2. Weight each by an exponential kernel on its distance from x.
  3. Fit a weighted ridge regression; report its coefficients.

Theory

LIME solves

\xi(x) = \arg\min_{g \in G} \ \mathcal{L}(f, g, \pi_x) + \Omega(g)

over interpretable models g, with locality supplied by the kernel

\pi_x(z) = \exp\left( -\frac{d(x, z)^2}{\sigma^2} \right)

Distances are measured in units of the background's standard deviation, so a feature measured in millions does not swamp one measured in units.

Parameters

estimator
Algorithm
A fitted model. Only predict or predict_proba is used.
x
np.ndarray of shape (n_features,)
The single sample to explain.
background
np.ndarray of shape (n_background, n_features)
Data defining the perturbation distribution and the feature scales.
class_index
int
Which class probability to explain, for a classifier.
n_samples
int = 2000
Perturbations drawn. More is steadier and slower.
kernel_width
float
Locality width in standardised units. Defaults to :math:`0.75 \sqrt{d}`, the usual heuristic.
n_features
int
Report only this many coefficients, chosen by magnitude. None keeps all.
feature_names
list of str
Names for the report.
random_state
int
Seed for the perturbations.

Returns

explanation
Explanation
values holds one coefficient per feature. metadata carries intercept, local_r2 — the surrogate's weighted fit quality — and prediction.

Notes

Complexity. One prediction pass over n_samples rows, plus a least-squares solve. Explaining a whole dataset means repeating that per row, which is why LIME is a tool for inspecting individual decisions rather than summarising a model.

Check local_r2 before believing the coefficients. They describe the surrogate, and if the surrogate fits badly the explanation describes nothing. A low value means the model is not locally linear at this point, and the honest response is to say so rather than to read the coefficients anyway.

LIME is not additive and its explanations are not stable. Unlike TreeExplainer, coefficients carry no guarantee of summing to the prediction, and re-running with a different seed can shift them — markedly so when n_samples is small or the kernel is narrow. If the model is a TuiML tree or forest, prefer TreeExplainer: it is exact, deterministic and cheaper.

Perturbing features independently also fabricates rows the data never contains, the same criticism that applies to partial_dependence.

References

Ribeiro2016
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). "Why Should I Trust You?": Explaining the Predictions of Any Classifier. ACM SIGKDD, 1135-1144. :doi:`10.1145/2939672.2939778`

See Also

TreeExplainer

Exact and deterministic, for tree models.
counterfactual

What would have to change, rather than what mattered.
python 
>>> import numpy as np
>>> from tuiml.explain import lime_explain
>>> from tuiml.algorithms.trees import RandomForestRegressor
>>> rng = np.random.default_rng(0)
>>> X = rng.normal(size=(300, 4))
>>> y = 5.0 * X[:, 2] + rng.normal(0, 0.1, 300)
>>> model = RandomForestRegressor(n_estimators=40, random_state=0).fit(X, y)
>>> result = lime_explain(model, X[0], background=X, random_state=0)
>>> result.top(1)[0][0]
'feature_2'
>>> bool(result.metadata['local_r2'] > 0.5)      # the surrogate fits
True
Func

counterfactual

Line 257
counterfactual(estimator: Any, x: np.ndarray, background: np.ndarray, target: Any, feature_names: Optional[List[str]]=None, max_features_changed: Optional[int]=None) -> Explanation

Find the smallest change that would flip the prediction.

Importance tells you what mattered. A counterfactual tells you what to do: "this loan would have been approved with £4,000 more income." That is the form an explanation has to take when someone is entitled to act on it, and it is the form regulation increasingly asks for.

Overview

  1. Find the nearest background sample the model already assigns to the
target class — a real, achievable point rather than an invented one.
  1. Walk each of its differing features back towards x, keeping a
change only while the prediction stays at the target.
  1. What survives is a sparse, minimal set of changes.
Anchoring on a real observation is deliberate. Optimising freely in feature space produces mathematically minimal counterfactuals that are physically impossible — negative ages, a house with two rooms and 400 square metres. Starting from data the world actually produced keeps the answer plausible.

Parameters

estimator
Algorithm
A fitted classifier exposing predict.
x
np.ndarray of shape (n_features,)
The sample whose prediction should change.
background
np.ndarray of shape (n_background, n_features)
Pool of realistic points to search.
target
Any
Desired predicted label.
feature_names
list of str
Names for the report.
max_features_changed
int
Keep at most this many changes, largest first. Fewer changes are easier to act on, at the cost of a larger move in each.

Returns

explanation
Explanation
values holds the change required per feature, zero where nothing need change. metadata carries the full counterfactual row, n_changed, distance and found.

Notes

Complexity. One prediction pass over the background, then at most one per differing feature.

A counterfactual is not advice. It says the model would decide differently, not that acting on it would change the real outcome — those coincide only if the model is causal, which it generally is not. It also describes one route among many; a different search returns a different, equally valid answer.

found is False when no background sample is classified as the target, which usually means the class is absent from the background rather than that no counterfactual exists.

References

Wachter2018
Wachter, S., Mittelstadt, B., & Russell, C. (2018). Counterfactual Explanations without Opening the Black Box. Harvard Journal of Law & Technology, 31(2), 841-887. :doi:`10.2139/ssrn.3063289`
Poyiadzi2020
Poyiadzi, R., Sokol, K., Santos-Rodriguez, R., De Bie, T., & Flach, P. (2020). FACE: Feasible and Actionable Counterfactual Explanations. AAAI/ACM AIES, 344-350. :doi:`10.1145/3375627.3375850`

See Also

lime_explain

What mattered, rather than what to change.
TreeExplainer

Exact attributions for tree models.
python 
>>> import numpy as np
>>> from tuiml.explain import counterfactual
>>> from tuiml.algorithms.trees import DecisionTreeClassifier
>>> rng = np.random.default_rng(0)
>>> X = rng.normal(size=(300, 3))
>>> y = (X[:, 0] > 0).astype(int)          # only feature 0 decides
>>> model = DecisionTreeClassifier(max_depth=4).fit(X, y)
>>> negative = X[model.predict(X) == 0][0]
>>> result = counterfactual(model, negative, background=X, target=1)
>>> bool(result.metadata['found'])
True
>>> int(result.metadata['n_changed'])      # one feature suffices
1