API Reference / algorithms / glassbox /

rulefit.py

RuleFit: a forest distilled into a sparse linear model of readable rules.

RuleFit fits a random forest, walks every root-to-leaf path to collect conjunctions of the form feature <= threshold / feature > threshold, and then fits a sparse linear model (L1 / L2 penalised least squares via coordinate descent) over the rules plus the original features. The result is a small, human-readable set of rules with coefficients, keeping the predictive power of the forest but with linear-model interpretability.

Classes

RuleFitRegressor

class algorithms.glassbox.rulefit.RuleFitRegressor(Regressor, _RuleFitBase)

RuleFit regressor: forest rules distilled into a sparse linear model.

A glassbox regressor that fits a random forest, extracts human-readable conjunctions from its root-to-leaf paths, and then fits a sparse linear model over those rules plus the original features.
Constructor 
__init__(
    self,
    n_estimators: int = 100,
    tree_size: int = 3,
    max_rules: Optional[int] = None,
    penalty: str = 'l1',
    alpha: float = 0.1,
    max_iter: int = 1000,
    tol: float = 0.0001,
    random_state: Optional[int] = None,
    feature_names: Optional[List[str]] = None,
)

Overview

  1. Fit a RandomForestRegressor
  2. Walk each tree, turning every root-to-leaf path into a conjunction of
feature <= t / feature > t conditions (a rule)
  1. Deduplicate rules, optionally capping to the most frequent
  2. Build a design matrix of rule indicators plus the raw features
  3. Fit a penalised linear model (L1 / L2) over that design matrix via
coordinate descent

Theory

The prediction is the linear form

\hat{y}(x) = \beta_0 + \sum_{r} a_r \cdot r(x) + \sum_{j} b_j x_j

where r(x) \in \{0, 1\} is a rule indicator. The coefficients are found by minimising

\min_{a, b} \frac{1}{2}\|\hat{y} - y\|^2 + \alpha \cdot l1\_ratio \|(a, b)\|_1 + \frac{\alpha (1 - l1\_ratio)}{2} \|(a, b)\|^2_2

so an L1 penalty drives most rule coefficients to exactly zero, leaving a small readable rule set.

Parameters

n_estimators
int = 100
Number of trees in the underlying forest.
tree_size
int = 3
Maximum depth of each tree, which bounds rule length.
max_rules
int or None = None
Cap on the number of distinct rules kept (None = unlimited).
penalty
{'l1', 'l2'} = 'l1'
Sparse (lasso) or ridge regularisation for the linear model.
alpha
float = 0.1
Regularisation strength.
max_iter
int = 1000
Maximum coordinate-descent passes.
tol
float = 1e-4
Convergence tolerance on the maximum weight change.
random_state
int or None = None
Seed for the forest (reproducible fits).
feature_names
list of str
Names used in the printed rules.

Attributes

rules_
list of str
Human-readable rule strings (deduplicated).
rule_coefs_
np.ndarray of shape (n_rules,)
Coefficient of each rule.
feature_coefs_
np.ndarray of shape (n_features,)
Coefficient of each original feature.
intercept_
float
Linear intercept.
n_rules_
int
Number of distinct rules extracted.
n_features_
int
Number of features seen during fit.

Notes

Complexity:

  • Training: forest fit O(T \cdot n \cdot \log n) plus
coordinate descent O(\text{max\_iter} \cdot n \cdot p) with p = rules + features.
  • Prediction: O(p) per sample.
When to use RuleFitRegressor:
  • When you want a small set of readable if feature > t rules
  • When the target has threshold effects a plain linear model misses
  • When you need sparse, auditable coefficients

References

Friedman2008
Friedman, J.H. and Popescu, B.E. (2008). Predictive learning via rule ensembles. The Annals of Applied Statistics, 2(3), 916-954. DOI: 10.1214/07-AOAS148

See Also

RuleFitClassifier

Classification counterpart.
RandomForestRegressor

The forest being distilled.
python 
>>> from tuiml.algorithms.glassbox import RuleFitRegressor
>>> import numpy as np
>>> X = np.array([[i] for i in range(40)], dtype=float)
>>> y = np.where(X.ravel() < 20.0, 0.0, 5.0)
>>> reg = RuleFitRegressor(n_estimators=50, tree_size=2, random_state=0)
>>> _ = reg.fit(X, y)
>>> float(np.abs(reg.predict(np.array([[25.0]]))[0] - 5.0)) < 1.5
True
>>> isinstance(reg.get_rules(), list) and len(reg.get_rules()) > 0
True

Methods

get_parameter_schema (cls) -> Dict[str, Dict[str, Any]]

Return JSON Schema for constructor parameters.

get_capabilities (cls) -> List[str]

Return regressor capabilities.

get_complexity (cls) -> str

Return time/space complexity.

get_references (cls) -> List[str]

Return academic references.

fit (self, X: np.ndarray, y: np.ndarray) -> 'RuleFitRegressor'

Fit the rule ensemble.

Parameters
X
np.ndarray of shape (n_samples, n_features)
Training features.
y
np.ndarray of shape (n_samples,)
Target values.
Returns
self
RuleFitRegressor
Fitted regressor.
predict (self, X: np.ndarray) -> np.ndarray

Predict target values.

Parameters
X
np.ndarray of shape (n_samples, n_features)
Input samples.
Returns
y_pred
np.ndarray of shape (n_samples,)
Predicted values.
score (self, X: np.ndarray, y: np.ndarray) -> float

Return the R-squared score on the given data.

Parameters
X
np.ndarray of shape (n_samples, n_features)
Test samples.
y
np.ndarray of shape (n_samples,)
True target values.
Returns
r2
float
R-squared score.
__repr__ (self) -> str

RuleFitClassifier

class algorithms.glassbox.rulefit.RuleFitClassifier(Classifier, _RuleFitBase)

RuleFit classifier: forest rules distilled into a sparse linear model.

A glassbox binary classifier that fits a random forest, extracts human-readable conjunctions from its root-to-leaf paths, and fits a sparse linear model over those rules plus the original features. The linear score is interpreted as a class-1 probability (linear probability model).
Constructor 
__init__(
    self,
    n_estimators: int = 100,
    tree_size: int = 3,
    max_rules: Optional[int] = None,
    penalty: str = 'l1',
    alpha: float = 0.1,
    max_iter: int = 1000,
    tol: float = 0.0001,
    random_state: Optional[int] = None,
    feature_names: Optional[List[str]] = None,
)

Overview

  1. Fit a RandomForestClassifier
  2. Walk each tree, turning every root-to-leaf path into a conjunction of
feature <= t / feature > t conditions (a rule)
  1. Deduplicate rules, optionally capping to the most frequent
  2. Encode the binary labels as 0/1 and build a design matrix of rule
indicators plus the raw features
  1. Fit a penalised linear model (L1 / L2) over that design matrix via
coordinate descent and interpret the score as a class-1 probability

Theory

The score is the linear form

s(x) = \beta_0 + \sum_{r} a_r \cdot r(x) + \sum_{j} b_j x_j

interpreted as the class-1 probability (a linear probability model, the approach of the original RuleFit paper), clipped to [0, 1]. Coefficients minimise a penalised least-squares objective on the 0/1 target, with L1 shrinkage yielding a sparse, readable rule set.

Parameters

n_estimators
int = 100
Number of trees in the underlying forest.
tree_size
int = 3
Maximum depth of each tree, which bounds rule length.
max_rules
int or None = None
Cap on the number of distinct rules kept (None = unlimited).
penalty
{'l1', 'l2'} = 'l1'
Sparse (lasso) or ridge regularisation for the linear model.
alpha
float = 0.1
Regularisation strength.
max_iter
int = 1000
Maximum coordinate-descent passes.
tol
float = 1e-4
Convergence tolerance on the maximum weight change.
random_state
int or None = None
Seed for the forest (reproducible fits).
feature_names
list of str
Names used in the printed rules.

Attributes

rules_
list of str
Human-readable rule strings (deduplicated).
rule_coefs_
np.ndarray of shape (n_rules,)
Coefficient of each rule.
feature_coefs_
np.ndarray of shape (n_features,)
Coefficient of each original feature.
intercept_
float
Linear intercept.
classes_
np.ndarray
The two class labels in sorted order.
n_rules_
int
Number of distinct rules extracted.
n_features_
int
Number of features seen during fit.

Notes

Complexity:

  • Training: forest fit O(T \cdot n \cdot \log n) plus
coordinate descent O(\text{max\_iter} \cdot n \cdot p) with p = rules + features.
  • Prediction: O(p) per sample.
When to use RuleFitClassifier:
  • Binary classification where a small readable rule set is required
  • When the decision boundary has threshold effects a logistic model misses
  • When you want sparse, auditable coefficients

References

Friedman2008
Friedman, J.H. and Popescu, B.E. (2008). Predictive learning via rule ensembles. The Annals of Applied Statistics, 2(3), 916-954. DOI: 10.1214/07-AOAS148

See Also

RuleFitRegressor

Regression counterpart.
RandomForestClassifier

The forest being distilled.
python 
>>> from tuiml.algorithms.glassbox import RuleFitClassifier
>>> import numpy as np
>>> X = np.array([[i] for i in range(40)], dtype=float)
>>> y = np.where(X.ravel() < 20.0, 0, 1)
>>> clf = RuleFitClassifier(n_estimators=50, tree_size=2, random_state=0)
>>> _ = clf.fit(X, y)
>>> clf.predict(np.array([[5.0], [35.0]])).tolist()
[0, 1]
>>> isinstance(clf.get_rules(), list) and len(clf.get_rules()) > 0
True

Methods

get_parameter_schema (cls) -> Dict[str, Dict[str, Any]]

Return JSON Schema for constructor parameters.

get_capabilities (cls) -> List[str]

Return classifier capabilities.

get_complexity (cls) -> str

Return time/space complexity.

get_references (cls) -> List[str]

Return academic references.

fit (self, X: np.ndarray, y: np.ndarray) -> 'RuleFitClassifier'

Fit the rule ensemble.

Parameters
X
np.ndarray of shape (n_samples, n_features)
Training features.
y
np.ndarray of shape (n_samples,)
Target labels (must be exactly two classes).
Returns
self
RuleFitClassifier
Fitted classifier.
predict (self, X: np.ndarray) -> np.ndarray

Predict class labels.

Parameters
X
np.ndarray of shape (n_samples, n_features)
Input samples.
Returns
y_pred
np.ndarray of shape (n_samples,)
Predicted class labels.
predict_proba (self, X: np.ndarray) -> np.ndarray

Predict class probabilities.

Parameters
X
np.ndarray of shape (n_samples, n_features)
Input samples.
Returns
proba
np.ndarray of shape (n_samples, 2)
Probabilities for classes 0 and 1.
__repr__ (self) -> str