API Reference / base /

metrics.py

Base classes and utility functions for evaluation metrics.

Metrics in TuiML provide a unified interface for assessing model performance across classification, regression, and clustering tasks.

Classes

MetricType

class base.metrics.MetricType(Enum)

Enumeration of machine learning task categories.

Used to validate if a metric is appropriate for a given model type.

AverageType

class base.metrics.AverageType(Enum)

Strategies for aggregating multi-class performance.

  • MICRO: Total true positives, false negatives and false positives.
  • MACRO: Unweighted mean per class (treats all classes equally).
  • WEIGHTED: Average weighted by class support (accounts for imbalance).
  • BINARY: Specific to problems with only two classes.

Metric

class base.metrics.Metric(ABC)

Abstract base class for all performance evaluators.

Metrics are callable objects that calculate a score comparing the ground truth (y_{true}) with the model predictions (y_{pred}).
Constructor 
__init__(
    self,
    name: str,
    metric_type: MetricType,
)

Methods

__init__ (self, name: str, metric_type: MetricType)

Initialize a metric.

Parameters
name
str
Name of the metric.
metric_type
MetricType
Task category the metric applies to (classification, regression, etc.).
compute (self, y_true: np.ndarray, y_pred: np.ndarray, **kwargs) -> float

Compute the metric value.

Parameters
y_true
np.ndarray of shape (n_samples,)
Ground truth labels.
y_pred
np.ndarray of shape (n_samples,)
Predicted labels or probabilities.
**kwargs
dict
Additional metric-specific parameters.
Returns
score
float
The computed metric value.
__call__ (self, y_true: np.ndarray, y_pred: np.ndarray, **kwargs) -> float

Compute the metric by calling the object directly.

Parameters
y_true
np.ndarray of shape (n_samples,)
Ground truth labels.
y_pred
np.ndarray of shape (n_samples,)
Predicted labels or probabilities.
**kwargs
dict
Additional metric-specific parameters.
Returns
score
float
The computed metric value.
__repr__ (self) -> str

Return string representation of the metric.

Functions

Func

check_consistent_length

Line 99
check_consistent_length(*arrays) -> None

Check that all arrays have consistent first dimensions.

Parameters

*arrays
sequence of array-like
Arrays to check. None entries are ignored.

Returns

None

Raises

ValueError
If arrays have inconsistent lengths.
Func

check_classification_targets

Line 120
check_classification_targets(y_true: np.ndarray, y_pred: np.ndarray) -> None

Check that y_true and y_pred are valid classification targets.

Parameters

y_true
np.ndarray of shape (n_samples,)
Ground truth labels.
y_pred
np.ndarray of shape (n_samples,)
Predicted labels.

Returns

None

Raises

ValueError
If inputs are empty or have inconsistent lengths.
Func

get_num_classes

Line 144
get_num_classes(y_true: np.ndarray, y_pred: Optional[np.ndarray]=None) -> int

Get the number of unique classes in the data.

Parameters

y_true
np.ndarray of shape (n_samples,)
Ground truth labels.
y_pred
np.ndarray of shape (n_samples,)
Predicted labels. If given, classes from both arrays are counted.

Returns

n_classes
int
Number of unique classes.
Func

get_class_labels

Line 165
get_class_labels(y_true: np.ndarray, y_pred: Optional[np.ndarray]=None) -> np.ndarray

Get sorted unique class labels from the data.

Parameters

y_true
np.ndarray of shape (n_samples,)
Ground truth labels.
y_pred
np.ndarray of shape (n_samples,)
Predicted labels. If given, labels from both arrays are included.

Returns

labels
np.ndarray
Sorted array of unique class labels.
Func

is_binary

Line 184
is_binary(y_true: np.ndarray, y_pred: Optional[np.ndarray]=None) -> bool

Check if this is a binary classification problem.

Parameters

y_true
np.ndarray of shape (n_samples,)
Ground truth labels.
y_pred
np.ndarray of shape (n_samples,)
Predicted labels.

Returns

binary
bool
True if the data contains exactly two classes.
Func

weighted_sum

Line 201
weighted_sum(values: np.ndarray, weights: np.ndarray) -> float

Compute the weighted average of values.

Parameters

values
np.ndarray
Values to aggregate.
weights
np.ndarray
Weight for each value.

Returns

result
float
Sum of values * weights normalized by the total weight.
Func

safe_divide

Line 218
safe_divide(numerator: Union[float, np.ndarray], denominator: Union[float, np.ndarray], zero_division: float=0.0) -> Union[float, np.ndarray]

Safely divide, handling division by zero.

Parameters

numerator
float or np.ndarray
Numerator.
denominator
float or np.ndarray
Denominator.
zero_division
float = 0.0
Value to return where the denominator is zero.

Returns

result
float or np.ndarray
Element-wise result of the division, with zero_division substituted wherever the denominator is zero.