TuiML vs scikit-learn vs Weka

Per-algorithm accuracy, R², training time, and memory across 51 real-world datasets from TabArena v0.1; every score is the mean over 10-fold cross-validation.

Results

Random Forest

● TuiML ● scikit-learn ● Weka · green = best on that dataset

Dataset	Size	TuiML	scikit-learn	Weka

Methodology

Protocol

[+]Datasets: 51 tabular datasets from TabArena v0.1 (38 classification, 13 regression)

[+]Frameworks: TuiML, scikit-learn, Weka (python-weka-wrapper3)

[+]Algorithms: only those available in all three libraries

[+]Validation: stratified 10-fold cross-validation for classification and shuffled 10-fold cross-validation for regression (fixed seed 42); fold assignments are identical across all three frameworks

[+]Preprocessing: attribute types from OpenML schema; numeric → median impute + standardize; nominal kept as discrete codes for Weka, one-hot for scikit-learn / TuiML; all fit on train only

[+]Hyperparameters: matched across libraries where the underlying model is the same; see algorithms.py for the full mapping. Defaults view also available.

[+]Metrics: Accuracy / R², training time, peak memory

Environment

[+]TuiML: v0.2.0

[+]scikit-learn: v1.8.0

[+]Weka: python-weka-wrapper3 0.3.2 (Java 21)

[+]Each experiment: isolated process, single-threaded