API Reference / algorithms / trees /

decision_stump.py

DecisionStumpClassifier classifier implementation.

Classes

DecisionStumpClassifier

class algorithms.trees.decision_stump.DecisionStumpClassifier(Classifier)

Decision Stump - a one-level decision tree.

A decision stump is a single-split decision tree that selects the best attribute and split point to minimize classification error. Decision stumps are commonly used as weak learners in ensemble methods such as AdaBoost and bagging.
Constructor 
__init__(
    self,
)

Overview

The algorithm builds a depth-1 tree:

  1. For each feature, evaluate all candidate split points
  2. For numeric features, find the threshold that minimizes weighted
misclassification error
  1. For nominal features, group attribute values by majority class
  2. Select the single best feature and split across all candidates
  3. Assign the majority class to each branch

Theory

For a numeric attribute with threshold t, the weighted error is:

E(t) = \frac{1}{W} \left( \sum_{x_j \leq t} w_j \cdot \mathbb{1}[y_j \neq \hat{y}_L] + \sum_{x_j > t} w_j \cdot \mathbb{1}[y_j \neq \hat{y}_R] \right)

where W = \sum w_j is the total weight, \hat{y}_L and \hat{y}_R are the majority classes of the left and right branches, and \mathbb{1}[\cdot] is the indicator function.

Parameters

(No user-configurable parameters.)

Attributes

feature_index_
int
Index of the feature used for splitting.
threshold_
float
Threshold value for numeric features.
is_numeric_
bool
Whether the split feature is numeric.
left_class_
Any
Class predicted when condition is True (value <= threshold).
right_class_
Any
Class predicted when condition is False (value > threshold).
left_distribution_
dict
Class distribution for the left branch.
right_distribution_
dict
Class distribution for the right branch.
classes_
np.ndarray
Unique class labels.
n_samples_
int
Number of training samples seen during fit().

Notes

Complexity:

  • Training: O(n \cdot m \cdot \log(n)) where n = samples,
m = features (sorting per feature)
  • Prediction: O(1) per sample (single comparison)
When to use DecisionStumpClassifier:
  • As a weak learner in boosting ensembles (e.g., AdaBoost)
  • When you need the simplest possible interpretable model
  • As a baseline classifier for benchmarking
  • When training speed is critical and high accuracy is not required

References

Iba1992
Iba, W. and Langley, P. (1992). Induction of One-Level Decision Trees. Proceedings of the 9th International Conference on Machine Learning, pp. 233-240.
Freund1997
Freund, Y. and Schapire, R.E. (1997). A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting. Journal of Computer and System Sciences, 55(1), pp. 119-139. DOI: 10.1006/jcss.1997.1504

See Also

RandomForestClassifier

Ensemble of randomized trees.

Basic usage as a standalone classifier:

python 
>>> from tuiml.algorithms.trees import DecisionStumpClassifier
>>> import numpy as np
>>>
>>> # Create sample data
>>> X = np.array([[1, 2], [3, 4], [5, 6], [7, 8]])
>>> y = np.array([0, 0, 1, 1])
>>>
>>> # Fit the stump
>>> clf = DecisionStumpClassifier()
>>> clf.fit(X, y)
DecisionStumpClassifier(...)
>>> predictions = clf.predict(X)

Methods

get_parameter_schema (cls) -> Dict[str, Dict[str, Any]]

Return parameter schema (DecisionStumpClassifier has no parameters).

get_capabilities (cls) -> List[str]

Return classifier capabilities.

get_complexity (cls) -> str

Return time/space complexity.

get_references (cls) -> List[str]

Return academic references.

fit (self, X: np.ndarray, y: np.ndarray, sample_weight: Optional[np.ndarray]=None) -> 'DecisionStumpClassifier'

Fit the DecisionStumpClassifier classifier.

Parameters
X
np.ndarray of shape (n_samples, n_features)
Training features.
y
np.ndarray of shape (n_samples,)
Target labels.
sample_weight
np.ndarray of shape (n_samples,)
Sample weights (for boosting applications).
Returns
self
DecisionStumpClassifier
Returns the fitted instance.
predict (self, X: np.ndarray) -> np.ndarray

Predict classes using the decision stump.

Parameters
X
np.ndarray of shape (n_samples, n_features)
Test features.
Returns
y_pred
np.ndarray of shape (n_samples,)
Predicted classes.
predict_proba (self, X: np.ndarray) -> np.ndarray

Predict class probabilities.

Parameters
X
np.ndarray of shape (n_samples, n_features)
Test features.
Returns
proba
np.ndarray of shape (n_samples, n_classes)
Class probabilities.
get_stump_description (self) -> str

Get a human-readable description of the decision stump.

Returns
description
str
Human-readable description of the stump rule.
__repr__ (self) -> str

String representation.