Evaluation metrics for uplift models.
Unlike a regression score, an uplift model is judged by how well it ranks individuals by their treatment effect: the treated individuals it places at the top should be the ones who gain the most from treatment. The Qini curve and its area (AUUC) measure exactly that.
Functions
qini_curve(uplift, treatment, y, normalize: bool=False) -> Tuple[np.ndarray, np.ndarray]
Compute the Qini curve: cumulative incremental gain when treating the top-ranked individuals.
Samples are ranked by predicted uplift (highest first). At each prefix of the ranking, the incremental gain is the outcome of the treated samples minus the outcome those samples would have produced as controls, where the control counterfactual is estimated by scaling the control outcomes by the global treatment ratio n_t / n_c.
Parameters
uplift
array-like of shape (n_samples,)
Predicted treatment effect for each sample.
treatment
array-like of shape (n_samples,)
Binary treatment indicator.
y
array-like of shape (n_samples,)
Numeric outcome.
normalize
bool
= False
If
True, divide the curve by its final value so it ends at 1.
Returns
x
np.ndarray of shape (n_samples + 1,)
Fraction of the population treated, from 0 to 1.
curve
np.ndarray of shape (n_samples + 1,)
Cumulative incremental gain at each prefix.
Raises
ValueError
If
treatment is not binary or is missing one of the two groups.
python
>>> import numpy as np
>>> from tuiml.algorithms.causal import qini_curve
>>> rng = np.random.RandomState(0)
>>> treatment = rng.randint(0, 2, size=200)
>>> y = treatment * 2.0 + rng.normal(0, 0.1, size=200)
>>> uplift = y # a perfect ranking correlates with the outcome
>>> x, curve = qini_curve(uplift, treatment, y)
>>> x.shape
(201,)auuc(uplift, treatment, y) -> float
Area under the uplift curve (Qini coefficient).
Computes the area under the Qini curve minus the area under the diagonal that a random ranking would trace. A value above zero means the model ranks high-uplift individuals ahead of low-uplift ones better than chance.
Parameters
uplift
array-like of shape (n_samples,)
Predicted treatment effect for each sample.
treatment
array-like of shape (n_samples,)
Binary treatment indicator.
y
array-like of shape (n_samples,)
Numeric outcome.
Returns
auuc
float
The Qini coefficient (area under the curve minus the random diagonal).
python
>>> import numpy as np
>>> from tuiml.algorithms.causal import auuc
>>> rng = np.random.RandomState(0)
>>> treatment = rng.randint(0, 2, size=400)
>>> y = treatment * 2.0 + rng.normal(0, 0.1, size=400)
>>> good = auuc(y, treatment, y)
>>> bad = auuc(rng.normal(0, 1, size=400), treatment, y)
>>> bool(good > bad)
Trueuplift_at_k(uplift, treatment, y, k: int=100) -> float
Mean uplift of the top-k predicted-treatment-effect group.
Ranks samples by predicted uplift and returns the difference between the mean outcome of the treated and control samples among the top
k.Parameters
uplift
array-like of shape (n_samples,)
Predicted treatment effect for each sample.
treatment
array-like of shape (n_samples,)
Binary treatment indicator.
y
array-like of shape (n_samples,)
Numeric outcome.
k
int
= 100
Number of top-ranked samples to consider.
Returns
uplift
float
mean(y | treated, top-k) - mean(y | control, top-k).
Raises
ValueError
If the top-
k group does not contain both treatment groups.
python
>>> import numpy as np
>>> from tuiml.algorithms.causal import uplift_at_k
>>> treatment = np.tile([0, 1], 200) # balanced groups
>>> y = treatment * 2.0 # treated outcome 2, control 0
>>> uplift = np.arange(400) # top-k = indices 300..399 (mixed)
>>> round(uplift_at_k(uplift, treatment, y, k=100), 1)
2.0