metrics/

Scoring functions for every task type.

Around seventy metrics, all as plain functions taking (y_true, y_pred). Anywhere TuiML accepts a metrics list — train, Benchmark, the MCP tools — the names come from here.

Classification

accuracy_score, balanced_accuracy_score, precision_score, recall_score, f1_score, fbeta_score, roc_auc_score, average_precision_score, log_loss, matthews_corrcoef, cohen_kappa_score, confusion_matrix, classification_report, plus the rate family (true_positive_rate, false_positive_rate, specificity_score, ...) and the curves (roc_curve, precision_recall_curve).

Regression

mean_squared_error, root_mean_squared_error, mean_absolute_error, r2_score, correlation_coefficient, and the relative errors (relative_absolute_error, root_relative_squared_error).

Clustering

silhouette_score, davies_bouldin_score, calinski_harabasz_score for unlabelled data; adjusted_rand_score, normalized_mutual_info_score, homogeneity_score, completeness_score, v_measure_score when true labels are known.

Information theory

entropy, information_gain, gain_ratio, mutual_information, symmetrical_uncertainty, kullback_leibler_divergence and friends, used for splitting criteria and feature ranking as well as evaluation.

Notes

On imbalanced data accuracy is misleading: predicting the majority class for everything already scores well. Prefer balanced_accuracy_score, f1_score or matthews_corrcoef there.

Multi-class variants take an average argument ("macro", "micro", "weighted"); "macro" weights every class equally, "weighted" by class frequency.

python

>>> from tuiml.evaluation.metrics import accuracy_score, f1_score
>>> y_true = [0, 1, 1, 0, 1]
>>> y_pred = [0, 1, 0, 0, 1]
>>> float(accuracy_score(y_true, y_pred))
0.8

Modules

classification

Classification evaluation metrics....

0 classes · 29 functions

clustering

Clustering evaluation metrics....

0 classes · 12 functions

feature_scoring

Univariate feature scoring for feature selection....

0 classes · 6 functions

information_theoretic

Information-theoretic evaluation metrics....

0 classes · 13 functions

regression

Regression evaluation metrics....

0 classes · 7 functions