API Reference / evaluation / visualization /

curves.py

Diagnostic curves for a single fitted model: ROC, precision-recall, and learning curves.

Where a metric collapses a model to one number, a curve shows the whole trade-off. This module covers the three curves worth plotting for most projects:

  • plot_roc_curve — true positive rate against false positive rate as the
decision threshold sweeps from strict to permissive. The area under it (AUC) is the probability that a random positive is scored above a random negative. Handles binary problems and, given a full probability matrix, multiclass one-vs-rest with a macro-average overlay.
  • plot_pr_curve — precision against recall over the same threshold
sweep, summarised by average precision. Prefer this to ROC on imbalanced data: ROC looks flatteringly good when negatives vastly outnumber positives, because a large absolute number of false positives is still a small false positive rate.
  • plot_learning_curve — train and validation score as the training set
grows. A validation curve still climbing at the right edge means more data would help; a wide, persistent gap between the two curves means overfitting, and two low curves converging means underfitting.

All three take arrays you already have — no model is refitted here. Each calls matplotlib.pyplot.show() and optionally writes a PNG via save_path. matplotlib is imported lazily, so every function raises ImportError when it is missing.

See Also

roc_auc_score

The scalar AUC without a plot.
average_precision_score

Scalar summary of the precision-recall curve.
plot_confusion_matrix

What happens at one specific threshold.

Functions

Func

plot_roc_curve

Line 96
plot_roc_curve(y_true: np.ndarray, y_score: np.ndarray, title: str='ROC Curve', figsize: Tuple[int, int]=(), save_path: str=None, show_auc: bool=True, label: str=None, show_grid: bool=False, classes: Optional[List]=None)

Plot a Receiver Operating Characteristic curve, binary or one-vs-rest.

The curve traces true positive rate against false positive rate as the decision threshold sweeps from "predict nothing positive" (bottom left) to "predict everything positive" (top right). A perfect ranker hugs the top-left corner; the dashed diagonal is what random guessing achieves. The shaded area under the curve is the AUC, and equals the probability that a randomly chosen positive sample is scored higher than a randomly chosen negative one, so 0.5 is chance and 1.0 is perfect.

The plot is threshold-free — it says how well the model ranks, not how well any particular cut-off classifies. Because the false positive rate is normalised by the number of negatives, ROC stays optimistic on heavily imbalanced data; pair it with plot_pr_curve there.

Two modes are selected automatically from the shape of y_score. With a 1-D score vector (or a 2-column probability matrix, whose second column is used) a single binary curve is drawn. With a probability matrix of three or more columns, one one-vs-rest curve is drawn per class plus a dotted macro-average curve interpolated on a common 200-point FPR grid.

Parameters

y_true
ndarray of shape (n_samples,)
True class labels, binary or multiclass. In the binary path, labels that are not already 0/1 are binarised against the largest label as the positive class.
y_score
ndarray of shape (n_samples,) or (n_samples, n_classes)
Positive-class probabilities (1-D) for binary problems, or per-class probabilities (2-D) as returned by predict_proba. Predicted labels will not produce a meaningful curve.
title
str = 'ROC Curve'
Axis title. Title-cased when rendered; ' (one-vs-rest)' is appended in the multiclass path.
figsize
tuple of (float, float), 6) = (8
Figure size in inches.
save_path
str
If given, the figure is written to this path as a 300-dpi PNG with a tight bounding box before being shown.
show_auc
bool = True
Append (AUC = ...) to the legend entry. Binary path only.
label
str
Legend text for the curve. Defaults to 'ROC'. Binary path only.
show_grid
bool = False
Currently has no effect — grid lines are always drawn.
classes
list
Class labels in the column order of y_score, used to label the per-class curves. Defaults to np.unique(y_true). Multiclass path only.

Returns

fpr
ndarray of shape (n_thresholds,)
False positive rates. In the multiclass path this is the shared 200-point grid the macro-average was computed on.
tpr
ndarray of shape (n_thresholds,)
Matching true positive rates, macro-averaged in the multiclass path.
auc_score
float
Area under the returned curve; the macro-average AUC in the multiclass path.

Raises

ImportError
If matplotlib is not installed (it is imported lazily).
ValueError
If y_true has more than two classes but y_score is 1-D, or if len(classes) does not match the number of y_score columns.

Notes

Side effects: mutates the global matplotlib style, calls matplotlib.pyplot.show() before returning, and writes a file when save_path is given. The figure object is not returned. Use a non-interactive backend such as Agg for headless rendering.

See Also

plot_pr_curve

Better view of the same model on imbalanced data.
roc_auc_score

The scalar AUC alone.
plot_confusion_matrix

Behaviour at one chosen threshold.
python 
>>> import numpy as np
>>> from tuiml.evaluation.visualization import plot_roc_curve
>>> y_true = np.array([0, 0, 1, 1, 0, 1])
>>> y_score = np.array([0.1, 0.4, 0.35, 0.8, 0.2, 0.9])
>>> fpr, tpr, auc_score = plot_roc_curve(y_true, y_score)   # doctest: +SKIP

Multiclass, straight from a fitted TuiML classifier:

python 
>>> from tuiml.datasets import load_iris
>>> from tuiml.evaluation.splitting import train_test_split
>>> from tuiml.algorithms import NaiveBayesClassifier
>>> X, y = load_iris()
>>> X_train, X_test, y_train, y_test = train_test_split(
...     X, y, test_size=0.3, random_state=0)
>>> clf = NaiveBayesClassifier().fit(X_train, y_train)
>>> proba = clf.predict_proba(X_test)
>>> fpr, tpr, macro_auc = plot_roc_curve(
...     y_test, proba, classes=[0, 1, 2],
...     save_path='roc.png')                               # doctest: +SKIP
Func

plot_pr_curve

Line 304
plot_pr_curve(y_true: np.ndarray, y_score: np.ndarray, title: str='Precision-Recall Curve', figsize: Tuple[int, int]=(), save_path: str=None, show_ap: bool=True, label: str=None, show_grid: bool=False)

Plot a precision-recall curve for a binary classifier.

Each point corresponds to one decision threshold: recall (x) is the fraction of true positives found, precision (y) is the fraction of positive predictions that were right. Lowering the threshold moves you right — more positives found — usually at the cost of precision, so the curve slopes down to the right. A model that is useful everywhere stays high across the whole width; a model that only works when it is very confident starts high and collapses.

The dashed horizontal baseline is the positive class prevalence, which is what a random classifier achieves. On imbalanced data that line sits low, which is exactly why this plot is more honest than ROC: beating it is a real achievement, and the gap above it is visible at a glance. The shaded area is summarised by the average precision (AP) reported in the legend.

Parameters

y_true
ndarray of shape (n_samples,)
Binary ground truth encoded as 0/1, where 1 is the positive class.
y_score
ndarray of shape (n_samples,)
Predicted probability or score for the positive class.
title
str = 'Precision-Recall Curve'
Axis title; title-cased when rendered.
figsize
tuple of (float, float), 6) = (8
Figure size in inches.
save_path
str
If given, the figure is written to this path as a 300-dpi PNG with a tight bounding box before being shown.
show_ap
bool = True
Append (AP = ...) to the legend entry.
label
str
Legend text for the curve. Defaults to 'PR'.
show_grid
bool = False
Currently has no effect — grid lines are always drawn.

Returns

recall
ndarray of shape (n_points,)
Recall values, sorted ascending.
precision
ndarray of shape (n_points,)
Precision at the same points.
ap
float
Average precision, computed as the trapezoidal area under the recall-sorted curve.

Raises

ImportError
If matplotlib is not installed (it is imported lazily).

Notes

The curve is evaluated at every distinct value in y_score, so cost is O(n^2) in the number of unique scores — subsample before plotting very large score vectors.

y_true must be 0/1: unlike plot_roc_curve, no binarisation of other label encodings is performed, and arbitrary labels silently yield an all-zero curve.

Side effects: mutates the global matplotlib style, calls matplotlib.pyplot.show() before returning, and writes a file when save_path is given. The figure object is not returned.

See Also

plot_roc_curve

The complementary threshold sweep.
average_precision_score

The scalar AP alone.
python 
>>> import numpy as np
>>> from tuiml.evaluation.visualization import plot_pr_curve
>>> y_true = np.array([0, 0, 1, 1, 0, 1, 0, 0])
>>> y_score = np.array([0.1, 0.4, 0.35, 0.8, 0.2, 0.9, 0.05, 0.3])
>>> recall, precision, ap = plot_pr_curve(y_true, y_score)   # doctest: +SKIP
Func

plot_learning_curve

Line 472
plot_learning_curve(train_sizes: np.ndarray, train_scores: np.ndarray, test_scores: np.ndarray, title: str='Learning Curve', figsize: Tuple[int, int]=(), save_path: str=None, metric_name: str='Score', show_std: bool=True, show_grid: bool=False)

Plot training and validation score as a function of training set size.

Two curves are drawn against the number of training samples: training score (circles) and cross-validation score (squares), each optionally with a \pm 1 standard deviation band across CV folds. This is the plot that tells you whether to collect more data or change the model:

  • Validation curve still rising at the right edge — more data will help.
  • Validation curve flat and a wide gap below the training curve —
overfitting; regularise or simplify.
  • Both curves flat, close together and low — underfitting; the model is
too simple or the features are too weak, and more data will not help.

Nothing is fitted here: you supply the sizes and the scores, typically collected by refitting an estimator on growing subsets and scoring each fit with a splitter from splitting.

Parameters

train_sizes
ndarray of shape (n_sizes,)
Number of training samples used at each point, in increasing order; used directly as the x coordinates.
train_scores
ndarray of shape (n_sizes,) or (n_sizes, n_splits)
Training scores. If 2-D, the mean over axis 1 is plotted and the standard deviation becomes the shaded band.
test_scores
ndarray of shape (n_sizes,) or (n_sizes, n_splits)
Validation/test scores, same shape convention as train_scores.
title
str = 'Learning Curve'
Axis title; title-cased when rendered.
figsize
tuple of (float, float), 6) = (10
Figure size in inches.
save_path
str
If given, the figure is written to this path as a 300-dpi PNG with a tight bounding box before being shown.
metric_name
str = 'Score'
Name of the metric, used as the y-axis label (e.g. 'Accuracy').
show_std
bool = True
Draw the standard deviation bands. Ignored for 1-D score arrays, which carry no spread.
show_grid
bool = False
Currently has no effect — grid lines are always drawn.

Returns

None
The figure is shown (and optionally saved) rather than returned.

Raises

ImportError
If matplotlib is not installed (it is imported lazily).

Notes

Side effects: mutates the global matplotlib style, calls matplotlib.pyplot.show(), and writes a file when save_path is given.

See Also

plot_roc_curve

Threshold behaviour of a single fitted model.
plot_boxplot_comparison

Score spread across algorithms rather than across training sizes.
python 
>>> import numpy as np
>>> from tuiml.evaluation.visualization import plot_learning_curve
>>> train_sizes = np.array([20, 40, 60, 80, 100])
>>> train_scores = np.array([[0.99, 0.98, 1.00],
...                          [0.97, 0.97, 0.98],
...                          [0.96, 0.96, 0.97],
...                          [0.96, 0.95, 0.96],
...                          [0.95, 0.95, 0.96]])
>>> test_scores = np.array([[0.72, 0.70, 0.75],
...                         [0.80, 0.79, 0.82],
...                         [0.85, 0.84, 0.86],
...                         [0.88, 0.87, 0.88],
...                         [0.89, 0.89, 0.90]])
>>> plot_learning_curve(train_sizes, train_scores, test_scores,
...                     metric_name='Accuracy')   # doctest: +SKIP