API Reference / foundation /

tabicl.py

TabICL — in-context learning on tabular data, with no gradient step.

Wraps the upstream tabicl package from Inria's Soda team. TuiML ships no weights; tabicl fetches its own checkpoint on first use, and both its code and its weights are BSD-3-Clause, the same license as TuiML.

Classes

TabICLClassifier

class foundation.tabicl.TabICLClassifier(_TabICLBase, Classifier)

Tabular classification by in-context learning, with no training.

TabICL is a pretrained foundation model: a transformer trained once, on millions of synthetic tabular tasks, to be a learning algorithm. Your training rows are fed to the frozen network as context alongside the rows you want predicted, and the answer falls out of a single forward pass. Nothing is optimised on your data — there is no gradient step, no hyperparameter search, and no fitted coefficient anywhere.
Constructor 
__init__(
    self,
    n_estimators: int = 8,
    softmax_temperature: float = 0.9,
    average_logits: bool = True,
    outlier_threshold: float = 4.0,
    batch_size: int = 8,
    device: Optional[str] = None,
    allow_auto_download: bool = True,
    model_path: Optional[str] = None,
    random_state: int = 42,
    n_jobs: Optional[int] = None,
    verbose: bool = False) -> None,
)

Overview

  1. Each column is normalised and outliers are clipped.
  2. A column-then-row attention stack embeds every cell in the context of
its column's distribution and its row's other values.
  1. Training rows and their labels form the context; test rows are
appended without labels.
  1. One forward pass returns a distribution over classes for each test row.
  2. Steps 1-4 repeat for n_estimators permutations of the features and
classes, and the logits are averaged.

Theory

A conventional classifier searches for parameters \theta minimising a loss on your data. TabICL does not. It approximates the posterior predictive distribution directly, having been pretrained to map a context set onto predictions:

p(y \mid x, D_{\text{train}}) = \int p(y \mid x, \theta) \, p(\theta \mid D_{\text{train}}) \, d\theta

The integral is what a Bayesian method would have to approximate by sampling. Pretraining on millions of synthetic datasets amortises it into the network's weights, so at your dataset the whole thing is one forward pass. This is why the method needs no tuning: the "learning algorithm" was itself learned, once, in advance.

Parameters

n_estimators
int = 8
Number of ensemble members, each with a different feature and class permutation. Logits are averaged.
softmax_temperature
float = 0.9
Temperature applied to the logits. Below 1 sharpens the predicted distribution; raise it if probabilities look overconfident.
average_logits
bool = True
Average ensemble members in logit space rather than probability space.
outlier_threshold
float = 4.0
Standard deviations beyond which a value is clipped in preprocessing.
batch_size
int = 8
Ensemble members evaluated per forward pass. Lower it before lowering n_estimators when memory is tight.
device
str or None = None
Torch device. None lets tabicl choose.
allow_auto_download
bool = True
Fetch the checkpoint (~150 MB) from the Hugging Face Hub on first use.
model_path
str or None = None
Local checkpoint path, bypassing the download.
random_state
int = 42
Seed for the ensemble permutations.
n_jobs
int or None = None
Preprocessing threads.
verbose
bool = False
Print progress, including the download.

Attributes

model_
object
The underlying tabicl.TabICLClassifier.
classes_
np.ndarray of shape (n_classes,)
Class labels seen in y.
n_features_in_
int
Number of features seen during fit.

Notes

Requires pip install 'tuiml[foundation]', which pulls in tabicl and PyTorch. Importing TuiML and constructing this class work without either; only fit needs them.

Weights are not shipped by TuiML. On first use tabicl downloads its checkpoint from the Hugging Face Hub into ~/.cache/huggingface. Code and weights are both BSD-3-Clause, matching TuiML's own license, so there is no commercial-use restriction to accept. That is not true of every tabular foundation model — several publish weights under non-commercial terms — which is why TabICL is the only one integrated here.

Complexity: fitting is O(1) in the sense that matters — no optimisation runs. Prediction is a transformer forward pass over n_{\text{train}} + n_{\text{test}} rows per ensemble member, so cost grows with the training set at predict time. A GPU is worth roughly an order of magnitude.

When to use TabICL:

  • Small to medium tabular data (up to roughly 100k rows) where you want a
strong result without tuning anything.
  • As a baseline that costs one line and no search budget.
  • When labelled data is scarce — amortised pretraining is at its most
valuable where a from-scratch model would overfit.

Prefer a gradient-boosted ensemble (XGBoostClassifier) on large data, where TabICL's per-prediction cost stops being worth it.

References

Qu2025
Qu, J., Holzmüller, D., Varoquaux, G. and Le Morvan, M. (2025). TabICL: A Tabular Foundation Model for In-Context Learning on Large Data. Proceedings of the 42nd International Conference on Machine Learning (ICML). arXiv: 2502.05564

See Also

TabICLRegressor

Regression counterpart.
FTTransformerClassifier

Native transformer trained on your data.
XGBoostClassifier

The usual strong tabular baseline.

Constructing and inspecting the model needs no optional dependency:

python 
>>> from tuiml.foundation import TabICLClassifier
>>> clf = TabICLClassifier(n_estimators=2)
>>> clf.n_estimators
2
>>> "softmax_temperature" in clf.get_parameter_schema()
True

Fitting requires pip install 'tuiml[foundation]' and downloads the

checkpoint on first use:

python 
>>> import numpy as np  # doctest: +SKIP
>>> rng = np.random.default_rng(0)  # doctest: +SKIP
>>> X = rng.normal(size=(60, 4))  # doctest: +SKIP
>>> y = (X[:, 0] > 0).astype(int)  # doctest: +SKIP
>>> clf.fit(X, y).predict(X[:5])  # doctest: +SKIP
array([1, 0, 0, 1, 1])

Methods

predict_proba (self, X: np.ndarray) -> np.ndarray

Return class probabilities from the frozen network.

Parameters
X
array-like of shape (n_samples, n_features)
Samples to score.
Returns
proba
np.ndarray of shape (n_samples, n_classes)
Per-class probabilities, rows summing to 1.
get_parameter_schema (cls) -> Dict[str, Dict[str, Any]]

Return JSON Schema for constructor parameters.

get_capabilities (cls) -> List[str]

Return the data and task properties this model supports.

TabICLRegressor

class foundation.tabicl.TabICLRegressor(_TabICLBase, Regressor)

Tabular regression by in-context learning, with no training.

The regression counterpart of TabICLClassifier, using the same frozen pretrained transformer and the same one-forward-pass prediction. See that class for the full description of the method; this docstring covers only what differs.
Constructor 
__init__(
    self,
    n_estimators: int = 8,
    outlier_threshold: float = 4.0,
    batch_size: int = 8,
    device: Optional[str] = None,
    allow_auto_download: bool = True,
    model_path: Optional[str] = None,
    random_state: int = 42,
    n_jobs: Optional[int] = None,
    verbose: bool = False) -> None,
)

Overview

  1. Columns are normalised and outliers clipped.
  2. Column-then-row attention embeds each cell.
  3. Training rows and their targets form the context; test rows are
appended without targets.
  1. One forward pass returns a predicted value per test row.
  2. Steps 1-4 repeat over n_estimators feature permutations and the
predictions are averaged.

Theory

As in the classifier, the network approximates the posterior predictive distribution rather than fitting parameters to your data:

p(y \mid x, D_{\text{train}}) = \int p(y \mid x, \theta) \, p(\theta \mid D_{\text{train}}) \, d\theta

with a continuous rather than categorical target. There is no class permutation to average over, so the regressor has neither class_shuffle_method nor softmax_temperature.

Parameters

n_estimators
int = 8
Ensemble members, each with a different feature permutation.
outlier_threshold
float = 4.0
Standard deviations beyond which a value is clipped in preprocessing.
batch_size
int = 8
Ensemble members per forward pass.
device
str or None = None
Torch device. None lets tabicl choose.
allow_auto_download
bool = True
Fetch the checkpoint (~150 MB) on first use.
model_path
str or None = None
Local checkpoint path, bypassing the download.
random_state
int = 42
Seed for the ensemble permutations.
n_jobs
int or None = None
Preprocessing threads.
verbose
bool = False
Print progress, including the download.

Attributes

model_
object
The underlying tabicl.TabICLRegressor.
n_features_in_
int
Number of features seen during fit.

Notes

Requires pip install 'tuiml[foundation]'. Importing and constructing work without it; only fit needs it. TuiML ships no weights — tabicl downloads its own checkpoint, BSD-3-Clause like TuiML itself.

Complexity: no optimisation runs at fit time; prediction is a forward pass over training plus test rows, per ensemble member.

When to use TabICLRegressor: small to medium tabular regression where you want a strong untuned baseline. Prefer XGBoostRegressor on large data, and NGBoostRegressor when you need a calibrated predictive distribution rather than a point.

References

Qu2025
Qu, J., Holzmüller, D., Varoquaux, G. and Le Morvan, M. (2025). TabICL: A Tabular Foundation Model for In-Context Learning on Large Data. Proceedings of the 42nd International Conference on Machine Learning (ICML). arXiv: 2502.05564

See Also

TabICLClassifier

Classification counterpart.
NGBoostRegressor

Native probabilistic boosting.
FTTransformerRegressor

Native transformer trained on your data.
python 
>>> from tuiml.foundation import TabICLRegressor
>>> reg = TabICLRegressor(n_estimators=2)
>>> reg.n_estimators
2
>>> "softmax_temperature" in reg.get_parameter_schema()
False

Methods

get_parameter_schema (cls) -> Dict[str, Dict[str, Any]]

Return JSON Schema for constructor parameters.

get_capabilities (cls) -> List[str]

Return the data and task properties this model supports.