API Reference / algorithms / ensemble /

gradient_boosting_regressor.py

GradientBoostingRegressor implementation.

Classes

GradientBoostingRegressor

class algorithms.ensemble.gradient_boosting_regressor.GradientBoostingRegressor(Regressor)

Gradient boosting for regression.

Builds an additive ensemble by iteratively fitting base regressors to the residuals (negative gradients) of the previous predictions.
Constructor 
__init__(
    self,
    base_regressor: Optional[Regressor] = None,
    n_estimators: int = 100,
    shrinkage: float = 0.1,
    minimize_absolute_error: bool = False,
)

Parameters

base_regressor
Regressor
The base regressor to use. If None, a simple decision stump is used.
n_estimators
int = 100
The number of boosting iterations.
shrinkage
float = 0.1
Learning rate/shrinkage factor.
minimize_absolute_error
bool = False
Whether to use L1 loss (MAE) instead of L2 loss (MSE).

Attributes

estimators_
list of Regressor
The collection of fitted base regressors.
initial_prediction_
float
The initial prediction (mean or median of target values).
python 
>>> from tuiml.algorithms.ensemble import GradientBoostingRegressor
>>> reg = GradientBoostingRegressor(n_estimators=50, shrinkage=0.1)
>>> reg.fit(X_train, y_train)
GradientBoostingRegressor(...)
>>> predictions = reg.predict(X_test)

References

1
Friedman, J. H. (2001). Greedy function approximation: A gradient boosting machine. Annals of Statistics, 29(5), 1189-1232.

Methods

get_parameter_schema (cls) -> Dict[str, Dict[str, Any]]

Return parameter schema.

get_capabilities (cls) -> List[str]

Return algorithm capabilities.

get_complexity (cls) -> str

Return time/space complexity.

get_references (cls) -> List[str]

Return academic references.

fit (self, X: np.ndarray, y: np.ndarray) -> 'GradientBoostingRegressor'

Fit the GradientBoostingRegressor model.

Parameters
X
np.ndarray of shape (n_samples, n_features)
Training features.
y
np.ndarray of shape (n_samples,)
Target values.
Returns
self
GradientBoostingRegressor
Returns the fitted instance.
predict (self, X: np.ndarray) -> np.ndarray

Predict target values for samples.

Parameters
X
np.ndarray of shape (n_samples, n_features)
Test features.
Returns
y_pred
np.ndarray of shape (n_samples,)
Predicted values.
score (self, X: np.ndarray, y: np.ndarray) -> float

Compute R-squared score.

Parameters
X
np.ndarray of shape (n_samples, n_features)
Test features.
y
np.ndarray of shape (n_samples,)
True target values.
Returns
score
float
R-squared score.
staged_predict (self, X: np.ndarray)

Predict at each boosting iteration.

Parameters
X
np.ndarray of shape (n_samples, n_features)
Test features.
Yields
y_pred
np.ndarray of shape (n_samples,)
Predictions at each stage.
__repr__ (self) -> str

String representation.