API Reference / algorithms / trees /

decision_tree.py

CART Decision Tree for classification and regression.

Classes

DecisionTreeClassifier

class algorithms.trees.decision_tree.DecisionTreeClassifier(Classifier)

CART decision tree for classification.

A CART (Classification and Regression Trees) classifier that uses vectorised batch prediction.
Constructor 
__init__(
    self,
    criterion: str = 'gini',
    max_depth: Optional[int] = None,
    min_samples_split: int = 2,
    min_samples_leaf: int = 1,
    min_impurity_decrease: float = 0.0,
    ccp_alpha: float = 0.0,
    random_state: Optional[int] = None,
)

Overview

  1. Build the tree recursively using the selected impurity criterion
(Gini, entropy, or log-loss) to evaluate candidate splits
  1. After fitting, flatten the tree into parallel arrays
  2. Predict via efficient tree traversal

Theory

Gini impurity (criterion="gini"):

G(S) = 1 - \sum_{k=1}^{C} p_k^2

Entropy / information gain (criterion="entropy" or "log_loss"):

H(S) = -\sum_{k=1}^{C} p_k \log_2 p_k

Gain ratio (criterion="gain_ratio", C4.5):

\text{GainRatio} = \frac{\Delta H}{\text{SplitInfo}}, \quad \text{SplitInfo} = -\sum_i \frac{|S_i|}{|S|} \log_2 \frac{|S_i|}{|S|}

where p_k is the proportion of class k in the node. A split is chosen to maximise the weighted impurity reduction (or gain ratio for C4.5):

\Delta I = I(S) - \frac{|S_L|}{|S|} I(S_L) - \frac{|S_R|}{|S|} I(S_R)

Optional minimal cost-complexity pruning removes branches whose effective \alpha is below ccp_alpha.

Parameters

criterion
str = "gini"
The function to measure the quality of a split. Supported criteria: "gini" for the Gini impurity (CART), "entropy" for the information gain (ID3), "log_loss" (alias for entropy), and "gain_ratio" for the C4.5 gain-ratio criterion.
max_depth
int or None = None
Maximum depth of the tree. None means unlimited.
min_samples_split
int = 2
Minimum samples required to split an internal node.
min_samples_leaf
int = 1
Minimum samples required at a leaf node.
min_impurity_decrease
float = 0.0
A node is split only if the impurity decrease is at least this value.
ccp_alpha
float = 0.0
Complexity parameter for minimal cost-complexity pruning. Subtrees with effective alpha less than ccp_alpha are pruned.
random_state
int or None = None
Random seed for reproducibility when features are tied.

Attributes

tree_
TreeNode
The root node of the recursive tree (training representation).
flat_tree_
FlattenedTree
Flattened parallel-array tree used for JIT prediction.
classes_
np.ndarray
Unique class labels discovered during fit().
n_classes_
int
Number of classes.
n_features_
int
Number of features seen during fit().
max_depth_
int
Actual depth of the fitted tree.
n_nodes_
int
Total number of nodes in the fitted tree.

Notes

Complexity:

  • Training: O(n \cdot m \cdot n \log n) where n = samples,
m = features (sorting per feature per node)
  • Prediction: O(d) per sample where d = tree depth, fully
vectorised across the batch

When to use DecisionTreeClassifier:

  • Large datasets where JIT-compiled splitting provides speedups
  • Batch prediction on GPU/TPU backends
  • When you need an interpretable single-tree model with hardware acceleration

References

Breiman1984
Breiman, L., Friedman, J., Olshen, R. and Stone, C. (1984). Classification and Regression Trees. Wadsworth International Group.

See Also

DecisionTreeRegressor

CART regression tree.
RandomForestClassifier

Ensemble of random trees.
python 
>>> from tuiml.algorithms.trees import DecisionTreeClassifier
>>> import numpy as np
>>>
>>> X = np.array([[1, 2], [3, 4], [5, 6], [7, 8], [2, 3], [4, 5]])
>>> y = np.array([0, 0, 1, 1, 0, 1])
>>>
>>> clf = DecisionTreeClassifier(max_depth=3)
>>> clf.fit(X, y)
DecisionTreeClassifier(max_depth=3, n_nodes=...)
>>> predictions = clf.predict(X)

Methods

get_parameter_schema (cls) -> Dict[str, Dict[str, Any]]

Return JSON Schema for constructor parameters.

get_capabilities (cls) -> List[str]

Return supported capabilities.

get_complexity (cls) -> str

Return complexity analysis.

get_references (cls) -> List[str]

Return academic citations.

fit (self, X: np.ndarray, y: np.ndarray) -> 'DecisionTreeClassifier'

Fit the CART classifier.

Parameters
X
np.ndarray of shape (n_samples, n_features)
Training feature matrix.
y
np.ndarray of shape (n_samples,)
Target class labels.
Returns
self
DecisionTreeClassifier
Fitted estimator.
predict (self, X: np.ndarray) -> np.ndarray

Predict class labels for samples in X.

Parameters
X
np.ndarray of shape (n_samples, n_features)
Input samples.
Returns
y_pred
np.ndarray of shape (n_samples,)
Predicted class labels.
predict_proba (self, X: np.ndarray) -> np.ndarray

Predict class probabilities for samples in X.

Parameters
X
np.ndarray of shape (n_samples, n_features)
Input samples.
Returns
proba
np.ndarray of shape (n_samples, n_classes)
Class probability estimates.
get_tree_description (self, node=None, depth: int=0) -> str

Return a human-readable text description of the tree.

Parameters
node
TreeNode or None
Starting node (defaults to tree_).
depth
int
Current indentation depth.
Returns
desc
str
Multi-line text representation.
__repr__ (self) -> str

Return string representation.

Returns
repr
str
String representation.

DecisionTreeRegressor

class algorithms.trees.decision_tree.DecisionTreeRegressor(Regressor)

CART decision tree for regression.

A CART regression tree with configurable splitting criteria (squared error, Friedman MSE, or absolute error) with vectorised batch prediction.
Constructor 
__init__(
    self,
    criterion: str = 'squared_error',
    max_depth: Optional[int] = None,
    min_samples_split: int = 2,
    min_samples_leaf: int = 1,
    min_impurity_decrease: float = 0.0,
    ccp_alpha: float = 0.0,
    random_state: Optional[int] = None,
)

Overview

  1. Build the tree recursively using the selected impurity criterion
(squared error, Friedman MSE, or absolute error) to evaluate splits
  1. After fitting, flatten the tree into parallel arrays
  2. Predict via efficient tree traversal

Theory

Squared error (criterion="squared_error"):

\text{MSE}(S) = \frac{1}{|S|} \sum_{i=1}^{|S|} (y_i - \bar{y})^2

Friedman MSE (criterion="friedman_mse"):

\Delta_{\text{friedman}} = \frac{n_L \cdot n_R}{n^2} (\bar{y}_L - \bar{y}_R)^2

Absolute error (criterion="absolute_error"):

\text{MAE}(S) = \frac{1}{|S|} \sum_{i=1}^{|S|} |y_i - \text{median}(y)|

Each leaf stores the mean (squared error, Friedman MSE) or median (absolute error) of its training targets.

Parameters

criterion
str = "squared_error"
The function to measure the quality of a split. Supported criteria: "squared_error" for variance reduction (CART), "friedman_mse" for Friedman's improvement score (better for boosting), and "absolute_error" for mean absolute error using median predictions.
max_depth
int or None = None
Maximum depth of the tree. None means unlimited.
min_samples_split
int = 2
Minimum samples required to split an internal node.
min_samples_leaf
int = 1
Minimum samples required at a leaf node.
min_impurity_decrease
float = 0.0
A node is split only if the impurity decrease is at least this value.
ccp_alpha
float = 0.0
Complexity parameter for minimal cost-complexity pruning.
random_state
int or None = None
Random seed for reproducibility when features are tied.

Attributes

tree_
TreeNode
The root node of the recursive tree (training representation).
flat_tree_
FlattenedTree
Flattened parallel-array tree used for JIT prediction.
n_features_
int
Number of features seen during fit().
max_depth_
int
Actual depth of the fitted tree.
n_nodes_
int
Total number of nodes in the fitted tree.

Notes

Complexity:

  • Training: O(n \cdot m \cdot n \log n) where n = samples,
m = features
  • Prediction: O(d) per sample where d = tree depth, fully
vectorised across the batch

When to use DecisionTreeRegressor:

  • Large regression datasets where JIT-compiled criteria speed up training
  • Batch prediction on GPU/TPU backends
  • When you need an interpretable single-tree regression model with hardware
acceleration

References

Breiman1984
Breiman, L., Friedman, J., Olshen, R. and Stone, C. (1984). Classification and Regression Trees. Wadsworth International Group.

See Also

DecisionTreeClassifier

CART classifier.
RandomForestRegressor

Ensemble of random regression trees.
python 
>>> from tuiml.algorithms.trees import DecisionTreeRegressor
>>> import numpy as np
>>>
>>> X = np.array([[1, 2], [3, 4], [5, 6], [7, 8]])
>>> y = np.array([1.0, 2.0, 3.0, 4.0])
>>>
>>> reg = DecisionTreeRegressor(max_depth=3)
>>> reg.fit(X, y)
DecisionTreeRegressor(max_depth=3, n_nodes=...)
>>> predictions = reg.predict(X)

Methods

get_parameter_schema (cls) -> Dict[str, Dict[str, Any]]

Return JSON Schema for constructor parameters.

get_capabilities (cls) -> List[str]

Return supported capabilities.

get_complexity (cls) -> str

Return complexity analysis.

get_references (cls) -> List[str]

Return academic citations.

fit (self, X: np.ndarray, y: np.ndarray) -> 'DecisionTreeRegressor'

Fit the CART regressor.

Parameters
X
np.ndarray of shape (n_samples, n_features)
Training feature matrix.
y
np.ndarray of shape (n_samples,)
Target values.
Returns
self
DecisionTreeRegressor
Fitted estimator.
predict (self, X: np.ndarray) -> np.ndarray

Predict target values for samples in X.

Parameters
X
np.ndarray of shape (n_samples, n_features)
Input samples.
Returns
y_pred
np.ndarray of shape (n_samples,)
Predicted values.
score (self, X: np.ndarray, y: np.ndarray) -> float

Return the R-squared score on the given test data.

Parameters
X
np.ndarray of shape (n_samples, n_features)
Test samples.
y
np.ndarray of shape (n_samples,)
True target values.
Returns
r2
float
R-squared score.
get_tree_description (self, node=None, depth: int=0) -> str

Return a human-readable text description of the tree.

Parameters
node
TreeNode or None
Starting node (defaults to tree_).
depth
int
Current indentation depth.
Returns
desc
str
Multi-line text representation.
__repr__ (self) -> str

Return string representation.

Returns
repr
str
String representation.