OneVsRestClassifier implementation.
Classes
OneVsRestClassifier for multiclass decomposition of binary classifiers.
OneVsRestClassifier enables binary classifiers to handle multi-class problems using various decomposition strategies such as One-vs-All, One-vs-One, and Error-Correcting Output Codes.
Constructor
__init__( self, base_classifier: Any = 'SVC', method: str = 'ova', random_state: Optional[int] = None, )
Overview
The algorithm decomposes a K-class problem into binary sub-problems:
- One-vs-All (OvA): Train K binary classifiers, each separating
- One-vs-One (OvO): Train K(K-1)/2 binary classifiers, one for
- Error-Correcting Output Codes (ECOC): Assign a binary codeword to
Theory
One-vs-All (OvA):
Train classifiers h_k for k = 1, \ldots, K, where h_k separates class k from all other classes. Prediction:
H(x) = \arg\max_{k} \; f_k(x)
where f_k(x) is the confidence score of classifier h_k.
One-vs-One (OvO):
Train classifiers h_{ij} for all pairs (i, j) with i < j. Prediction uses voting:
H(x) = \arg\max_{k} \sum_{(i,j): i<j} \mathbb{1}[h_{ij}(x) = k]
ECOC:
Assign a code matrix M \in \{-1, +1\}^{K \times B} and train B binary classifiers. Prediction finds the nearest codeword:
H(x) = \arg\min_{k} \; \|M_k - f(x)\|^2
Parameters
base_classifier
str or class
= 'SVC'
The binary classifier to use as the base learner.
method
{'ova', 'ovo', 'ecoc'}
= 'ova'
The decomposition method:
-
'ova'- One-vs-All (trains :math:`K` classifiers) -
'ovo'- One-vs-One (trains :math:`K(K-1)/2` classifiers) -
'ecoc'- Error-Correcting Output Codes
random_state
int or None
= None
Random seed for reproducibility. Primarily used for ECOC code matrix generation.
Attributes
estimators_
list
The collection of fitted binary classifiers.
classes_
np.ndarray
The unique class labels discovered during
fit().
n_classes_
int
The number of distinct classes.
Notes
Complexity:
- Training (OvA): O(K \cdot n \cdot C_{\text{base}}) where K = number of classes
- Training (OvO): O\bigl(\frac{K(K-1)}{2} \cdot \frac{2n}{K} \cdot C_{\text{base}}\bigr)
- Training (ECOC): O(B \cdot n \cdot C_{\text{base}}) where B = number of code bits
- Prediction: O(M \cdot C_{\text{predict}}) per sample, where M = number of estimators
- When your base classifier only supports binary classification (e.g., SVM)
- When you need to extend a strong binary learner to multiclass settings
- ECOC is preferred when robustness to individual classifier errors is desired
- OvO is preferred for classifiers with high training cost (smaller subproblems)
References
Dietterich1995
Dietterich, T.G. and Bakiri, G. (1995).
Solving Multiclass Learning Problems via Error-Correcting
Output Codes.
Journal of Artificial Intelligence Research, 2, 263-286.
DOI: 10.1613/jair.105
Allwein2000
Allwein, E.L., Schapire, R.E. and Singer, Y. (2000).
Reducing Multiclass to Binary: A Unifying Approach for Margin
Classifiers.
Journal of Machine Learning Research, 1, 113-141.
Rifkin2004
Rifkin, R. and Klautau, A. (2004).
In Defense of One-Vs-All Classification.
Journal of Machine Learning Research, 5, 101-141.
See Also
Basic usage with One-vs-One decomposition:
python
>>> from tuiml.algorithms.ensemble import OneVsRestClassifier
>>> import numpy as np
>>>
>>> # Create sample multiclass data
>>> X_train = np.array([[1, 2], [2, 3], [3, 1], [4, 3], [5, 2], [6, 1]])
>>> y_train = np.array([0, 0, 1, 1, 2, 2])
>>>
>>> # Fit with One-vs-One method
>>> clf = OneVsRestClassifier(base_classifier='SVC', method='ovo')
>>> clf.fit(X_train, y_train)
OneVsRestClassifier(...)
>>> predictions = clf.predict(X_train)Methods
get_parameter_schema
(cls) -> Dict[str, Dict[str, Any]]
get_capabilities
(cls) -> List[str]
get_complexity
(cls) -> str
get_references
(cls) -> List[str]
__repr__
(self) -> str