tabum-v1.1

tabUM v1.1

A small tabular in-context-learning foundation model: 14.31M parameters, zero-shot classification and regression on any table, native missing-value handling, up to 100 classes and 2,000 features. MIT licensed, built clean-room (no third-party model code).

Code, notebook, and full results: github.com/heldernoid/tabum

What it does

tabUM never trains on your data. fit(X, y) stores your rows; predict(X_test) runs a single forward pass in which every test row attends to every training row (in-context learning). Pretraining happened once, on millions of synthetic tasks sampled from causal graphs, Gaussian processes, and random tree ensembles.

• prediction in seconds, no tuning, no preprocessing pipelines
• NaN is a first-class value: missingness is signal, never imputed
• calibrated probabilities; full regression distributions (any quantile) in one pass
• up to 100 classes (most tabular foundation models cap at 10)
• optional finetune() (~15s per-dataset adaptation) and explain() (feature importances + the exact training rows the prediction attended to)

Usage

pip install git+https://github.com/heldernoid/tabum

from huggingface_hub import snapshot_download
from tabum.model import TabUM
from tabum.inference import TabUMClassifier, TabUMRegressor

model = TabUM.from_pretrained(snapshot_download("helmo/tabum-v1.1"), device="cuda")

clf = TabUMClassifier(model=model, n_ensemble=8).fit(X_train, y_train)
proba = clf.predict_proba(X_test)

reg = TabUMRegressor(model=model, n_ensemble=8).fit(X_train, y_train)
p90 = reg.predict_quantile(X_test, 0.9)

Evaluation

TabArena-v0.1, all 51 datasets, identical protocol for every model (at most 2,000 train / 1,000 test rows, single split, seed 0; competitors run as black-box pip packages on the same splits). Per-dataset numbers are in the GitHub repository.

Strengths and honest limitations:

• beats fitted linear baselines everywhere, zero-shot
• finetuned, it ties default gradient boosting on classification (20/38 wins) and trails it by 1.8 R² points on regression
• 1.3 to 1.7 accuracy points behind TabPFN v2 and TabICL (the state of the art in this class), at 14M parameters and MIT license
• many-class is a differentiator: letter (26 classes) scores 0.79 vs 0.74 for fitted logistic regression; TabPFN-class models cap at 10 classes
• strongest regime: small noisy tables (under ~1,000 rows), where the learned prior beats boosting; on Titanic with 12% missing cells fed raw it beats an imputed+scaled logistic regression pipeline
• context lengths validated to 64k rows (accuracy keeps improving with more context)

Model details

Intended use and limitations

Intended for tabular prediction on datasets from roughly 50 to tens of thousands of rows, up to 2,000 features, up to 100 classes. Not a language model: free-text columns should be dropped or encoded upstream. Regression on large, low-noise datasets is better served by gradient boosting; tabUM's edge is speed-to-first-prediction, small data, many classes, missing values, and built-in explanations.