gradient_boosting/

• XGBoostClassifier / XGBoostRegressor: Regularised boosting; the

general-purpose default of the three.

• LightGBMClassifier / LightGBMRegressor: Leaf-wise growth with

histogram binning; fastest on wide or large data.

• CatBoostClassifier / CatBoostRegressor: Ordered boosting with native

categorical handling; strongest when categorical columns dominate.

• NGBoostClassifier / NGBoostRegressor: Natural gradient boosting of a

proper scoring rule; predicts a calibrated distribution, not just a mean.

XGBoost, LightGBM and CatBoost wrap the three upstream libraries, which are required dependencies of TuiML rather than optional extras. Unlike most of algorithms, those implementations are not native.

NGBoost is the exception: it is a native pure-NumPy implementation with no external dependency, boosting TuiML's own DecisionTreeRegressor base learners. It is also the only member of this module that predicts a full distribution rather than a point estimate.

Trees are scale-invariant, so feature scaling gains nothing here. Set random_state (or pass random_seed to train) for reproducible fits — the wrapped libraries sample rows and columns while boosting, and NGBoost subsamples rows when minibatch_frac < 1.