API Reference / explain /

global_.py

Global model explanations: a surrogate tree and feature-interaction strength.

Functions

Func

surrogate_tree

Line 39
surrogate_tree(estimator: Any, X: np.ndarray, max_depth: int=3, random_state: Optional[int]=None) -> Explanation

Approximate a model with a single readable decision tree.

An ensemble or a neural network cannot be read by a person. A shallow tree can. The surrogate trick is to train one on the model's own predictions: the tree learns not the true labels but the pattern of "what the black box says", so its splits describe the model's behaviour, which is the thing being explained. Because the surrogate is a genuine TuiML estimator, every tool that works on a tree — TreeExplainer, plotting, partial_dependence — then works on the explanation.

Overview

  1. Collect the model's predictions on X.
  2. Fit a decision tree to those predictions, keeping the depth small.
  3. Return the fitted tree; its accuracy on the predictions is the
fidelity.

Parameters

estimator
Algorithm
A fitted model, of any kind.
X
np.ndarray of shape (n_samples, n_features)
Data to trace the model over. Use a held-out set or a representative sample; the surrogate only explains behaviour the data exercises.
max_depth
int = 3
Depth of the surrogate. Three keeps it readable; deeper captures more of the model at the cost of unreadability, which defeats the purpose.
random_state
int
Seed for the tree.

Returns

explanation
Explanation
values holds nothing — the payload is metadata['tree'], the fitted surrogate. metadata['fidelity'] is the surrogate's R² or accuracy on the model's predictions.

Notes

Fidelity is the number that decides whether to trust it. A surrogate that reproduces the model's predictions faithfully is a fair summary of the model; one that does not is a fair summary of nothing. There is no universal threshold — check it against a plain baseline — but a surrogate barely better than guessing says the model has no shallow structure, which is itself a useful thing to learn.

The surrogate describes global behaviour where the data is dense and nothing at all where it is sparse. It will also, by construction, miss detail the shallow tree cannot represent — that is the price of readability, and it is exactly the trade TreeExplainer avoids by being per-sample and exact.

References

Craven1996
Craven, M., & Shavlik, J. W. (1996). Extracting Tree-Structured Representations of Trained Networks. NeurIPS, 24-30.
Ribeiro2016
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). "Why Should I Trust You?": Explaining the Predictions of Any Classifier. ACM SIGKDD, 1135-1144. :doi:`10.1145/2939672.2939778`

See Also

TreeExplainer

Exact per-sample attributions for tree models.
friedman_h_statistic

Quantifies a specific interaction, rather than summarising the whole model.
python 
>>> import numpy as np
>>> from tuiml.explain import surrogate_tree
>>> from tuiml.algorithms.trees import RandomForestRegressor
>>> rng = np.random.default_rng(0)
>>> X = rng.normal(size=(400, 4))
>>> y = 3.0 * X[:, 0] - X[:, 1] + rng.normal(0, 0.2, 400)
>>> model = RandomForestRegressor(n_estimators=60, random_state=0).fit(X, y)
>>> result = surrogate_tree(model, X, max_depth=3, random_state=0)
>>> result.metadata['fidelity'] > 0.8
True
>>> result.metadata['tree'].tree_ is not None
True
Func

friedman_h_statistic

Line 146
friedman_h_statistic(estimator: Any, X: np.ndarray, feature_a: int, feature_b: int, n_points: int=20, random_state: Optional[int]=None) -> float

Quantify how much two features interact, in the Friedman sense.

Most attribution methods attribute each feature its own share. An interaction is what is left after both have been credited — the part of the prediction that depends on the combination, in a way neither feature explains alone. Friedman's H-statistic measures that residual directly: it is 0 for a purely additive model and grows as the interaction does.

Overview

  1. Compute the two single-feature partial-dependence functions, which
capture what each feature explains on its own.
  1. Compute the joint partial-dependence surface, which captures both
together.
  1. H^2 is the variance in the joint surface left unexplained by the
sum of the two single ones.

Theory

With PD_{jk} the joint partial dependence and PD_j, PD_k the individual ones,

H_{jk}^2 = \frac{ \sum_i \left( \tilde{PD}_{jk} - \tilde{PD}_j - \tilde{PD}_k \right)^2 }{ \sum_i \tilde{PD}_{jk}^{\,2} }

where the tilde marks centred partial dependences, each shifted so its mean over the evaluation grid is zero. Centring matters: an uncentred model with a constant level c would report an interaction of H^2 = 1 between two features it does not use, because the flat joint surface c compared against two flat curves summing to 2c leaves a residual of -c.

evaluated over the grid of observed values. A model with no interaction has PD_{jk} = PD_j + PD_k exactly, so H^2 = 0; the more of the surface the two single curves fail to explain, the closer it gets to 1.

Parameters

estimator
Algorithm
A fitted model.
X
np.ndarray of shape (n_samples, n_features)
Data defining the grid and the rows averaged over.
feature_a, feature_b
int
The pair to test.
n_points
int = 20
Grid points per feature. Cost grows with its square.
random_state
int
Seed for downsampling large grids.

Returns

h_squared
float
Interaction strength in [0, 1]. Roughly: below 0.01 negligible, above 0.1 worth investigating. Values above 1 are possible when the single curves happen to anti-correlate with the joint surface; treat them as "strong", not as a violation.

Notes

Complexity. O(n^2_{\text{points}}) prediction passes over X, so keep n_points modest — 20 gives 400 passes.

A low value is an assumption confirmed, not a null result. H measures the interaction on the grid of observed values; an interaction confined to a region the data never visits is invisible to it. And like partial_dependence, the marginalisation can evaluate the model on feature combinations the data does not contain.

References

Friedman2008
Friedman, J. H., & Popescu, B. E. (2008). Predictive Learning via Rule Ensembles. Annals of Applied Statistics, 2(3), 916-954. :doi:`10.1214/07-AOAS148`

See Also

partial_dependence

The single-feature curves this method extends.
surrogate_tree

The whole-model counterpart.
python 
>>> import numpy as np
>>> from tuiml.explain import friedman_h_statistic
>>> from tuiml.algorithms.trees import RandomForestRegressor
>>> rng = np.random.default_rng(0)
>>> X = rng.normal(size=(400, 3))
>>> additive = 3.0 * X[:, 0] + 2.0 * X[:, 1] + rng.normal(0, 0.2, 400)
>>> interaction = 3.0 * X[:, 0] * X[:, 1] + rng.normal(0, 0.2, 400)
>>> add = RandomForestRegressor(n_estimators=40, random_state=0).fit(X, additive)
>>> inter = RandomForestRegressor(n_estimators=40, random_state=0).fit(X, interaction)
>>> h_add = friedman_h_statistic(add, X, 0, 1, n_points=8, random_state=0)
>>> h_inter = friedman_h_statistic(inter, X, 0, 1, n_points=8, random_state=0)
>>> bool(h_inter > h_add)
True