fela-tab

N.B. Check us out on Github if you want a prepackaged way to deploy this on Postgres. Google Sheets Extension coming soon!!!

DISCLAIMER

This model is a research preview. It is offered for advancing public science and for evaluation. It is not a substitute for a domain expert, and it is not a certified decision system.

FelaTab: point at any table in context tabular model

FelaTab reads a small table you already have and predicts the missing cells. You give it some example rows with their answers (the support rows) and one or more rows you want filled in (the query rows); it learns the pattern from your examples in a single pass and returns the answer with a calibrated confidence range. There is no per table training, no fitting, and no setup: you point it at a table and it predicts. It runs on a plain CPU with no GPU.

It is a prior fitted network (a "foundation model for tables"), trained on millions of synthetic tables so that it generalizes zero shot to real tables it has never seen.

What goes in, what comes out

• Input: one table, split into SUPPORT rows (features + a known label) and QUERY rows (features only). Features are plain numbers; the model standardizes them for you. Up to 100 feature columns and, for classification, up to 10 classes.
• Output:
  • Classification: a probability for each class on every query row (they sum to one). Pick the top class as the prediction and read the probability as the confidence.
  • Regression: a predicted value plus a standard deviation on every query row, an honest error bar in the original units of your label, not just a point estimate.
• In plain terms: "these rows look labelled like this; fill the blank ones and tell me how sure you are."

Why we built it this way

The "sequence" the model attends over is the SET of rows of your table. Support rows carry their label and write into a fixed size working state; query rows are read only (they never write the state and never see each other, so there is no test time leakage). The mixer is a delta rule linear attention (an overwrite update that fixes the recall weakness of plain linear attention) plus a variable chunk landmark attention over pooled support rows. Both are linear in the number of rows with a fixed working state, so the model scales past the quadratic roughly 10k row cap of full attention tabular models, and its memory stays flat as the support set grows. That is what lets it run on a low power CPU and index large tables without a GPU.

The two tiers (one model, pick your size)

FelaTab ships two tiers from ONE MatFormer nested training run. The small model is a strict top left prefix slice of the big model's weights (nesting verified bit exact at full granularity), so you get both from a single set of weights and can pick your size/accuracy tradeoff.

int8 is close to lossless here (see below), so the int8 files are the recommended deploy artifact. The in browser demo runs the small int8 tier.

Performance and footprint

Measured on CPU. The working memory is flat as the support set grows: on the OpenML adult dataset the big model's resident set stays at about 3.79 GB from 500 support rows all the way to 10,000 support rows (the model carries a fixed size state, not a growing key value cache), while a full attention model would grow quadratically. Row throughput is linear in the support size.

Accuracy (held out OpenML)

Protocol: zero shot, TabPFN style. The train split of a real OpenML dataset is fed as support rows, the test split as query rows, in one in context forward, no per dataset training. The reported battery is 8 classification and 5 regression datasets. Field bars are untuned scikit-learn and a TUNED LightGBM; the "full ensemble" is FelaTab combined with gradient boosted trees and ridge. The numbers below are measured on the shipped weights.

Classification (8 datasets, mean accuracy)

• With no tuning, FelaTab alone matches or beats a tuned LightGBM on 5 of the 8 datasets.
• FelaTab alone reaches about 97% of the full ensemble's accuracy (0.819 vs 0.840).
• Calibration is good: expected calibration error about 0.051 for FelaTab, and a conformal wrapper hits its target coverage (empirical coverage about 0.91 for FelaTab and about 0.92 for the ensemble at a 0.90 guarantee).
• Inference time bagging (averaging the frozen model over permuted feature orders and a reshuffled support set) did not improve the big tier on this battery (it measured 0.811, below the single pass), so the single pass is the number to use.

Head to head vs Google TabFM (zero shot) and gradient boosted trees

Independent battery: 8 OpenML classification datasets (adult, credit-g, blood-transfusion, churn, electricity, vehicle, segment, jungle_chess), stratified 80/20 split, seed 42. FelaTab and google/tabfm-1.0.0-pytorch run zero shot (train split as in context support, capped at 3000 rows); XGBoost / LightGBM are trained on the full train split. Deep learning models ran on a local GPU (ROCm), the GBMs on CPU. Reproduce with benchmark/benchmark.py.

Summary (mean across 8 datasets)

Efficiency (this is where FelaTab lives)

• FelaTab-small is ~66x lower latency per sample than TabFM (0.79 vs 52.4 ms) at roughly 1/32nd the parameter count (51.6M vs 1.64B), and it does that while also running comfortably on a plain CPU.
• TabFM is more accurate on this battery (it won 7 of 8 datasets on accuracy; FelaTab-big tied it on credit-g). FelaTab's trade: give up some accuracy, get an order of magnitude faster, smaller, CPU-native, zero-setup predictor.
• Full per-dataset tables (accuracy / ROC-AUC / log loss / F1) are in benchmark/model_card_snippet.md; raw numbers in benchmark/results.csv.

TabArena

FelaTab ran the full TabArena v0.1 suite (51 datasets, all CV splits, 2448 task-fits, 0% imputed) with benchmark/tabarena/run_tabarena.py against the official TabArena cached leaderboard (83 method configs). Classification is 100% native FelaTab, zero-shot; regression tasks are delegated to a LightGBM regressor inside the wrapper (composite system — FelaTab's native regression head benchmarks below default RandomForest today; the delegate mirrors how the model ships in production).

The bagged config (p-value 0.044) permutes row + column order over n_bag=8 support subsets and averages the predictions — measured +1.4 AUC on adult over a single pass. An earlier 2-config run (big+small, 1632 tasks) landed 913 / 896 and is still in the results repo (eval/felatab_full/).

Reference points from the same leaderboard: XGBoost (default) 1203, RandomForest (default) 1000, Linear (tuned) 938, KNN (tuned) 890; TabPFNv2-family and tuned GBMs sit at 1300–1900 on GPU. FelaTab's pitch is not the Elo column: it is the only zero-shot entry that runs CPU-native (everything above it is GPU or trained per-dataset), at 1/32 the parameter count of comparable tabular FMs, with the int8 tier deployable in-browser (WASM) and inside Postgres.

Raw per-task results, the full leaderboard table, and reproducibility scripts live in lowdown-labs/fela-tab-tabarena-results (raw/ = per-fold results.pkl, eval/felatab_full/ = merged leaderboard + figures). Reproduce with benchmark/tabarena/launch_ec2.py + ec2_run.sh (~$70 on one c7i.8xlarge, sharded across two).

Regression (5 datasets)

On regression accuracy FelaTab alone beats neither untuned scikit-learn nor a tuned LightGBM on any of the 5 datasets (0 of 5) but it's not off by far; its mean R2 is 0.813 versus 0.880 for a tuned LightGBM.

This is a training budget limit of the current model, not a bug. What FelaTab ships for regression is, however, calibrated uncertainty: a Gaussian mean and standard deviation per row, plus a conformal wrapper that reaches its coverage target (empirical coverage about 0.91 at a 0.90 guarantee). If you need maximum regression accuracy, use a gradient boosted tree; use FelaTab's regression head for a fast zero shot estimate with a trustworthy error bar.

int8 is close to lossless

Dynamic int8 on the bulk linear layers barely moves accuracy: classification 0.819 fp32 vs 0.816 int8 for the big tier and 0.810 vs 0.809 for the small tier, regression within 0.002 R2 either way. The int8 files are 4x smaller and are the recommended deploy format.

How to run it

See quickstart/ for a runnable example on public sklearn datasets. The short version:

from modeling import load_model, predict

model = load_model("/path/to/repo_dir", tier="small")   # or "big"; or a HF repo id
# Xtr, ytr = your labelled support rows;  Xte = the rows to predict
probs = predict(model, Xtr, ytr, Xte, task="classification", n_classes=3)   # [n_query, 3]
mean, std = predict(model, Xtr, ytr, Xte, task="regression")                # error bars

The loader ships config_big.json / config_small.json (a top level config.json mirrors the big tier) and a self contained modeling.py with load_model / from_pretrained. It auto detects the int8 bundles and dequantizes them, so you always get a plain fp32 model back. For an interactive playground, see the Hugging Face Space in space/.

Serving artifacts

• model_<tier>.safetensors (fp32) and model_<tier>_int8.safetensors (int8), plus the matching config_<tier>.json.
• verify.py runs two fixed bundled tasks (one classification, one regression), checks the output shapes, that the regression standard deviations are positive, and that the small tier predictions match a stored reference within the shipped int8 tolerance.
• modeling.py also exposes predict_bagged (the inference bagging path) and the full architecture for standard tooling.

FelaTab is embedded natively in the CPU FELA server and in a PostgreSQL extension (fela_classify in database), and runs client side in the browser via WebAssembly (the small int8 tier). Those paths reproduce the Python predictions to within the int8 tolerance.

Training data

FelaTab is a prior fitted network: trained on synthetic tables from a structural causal model prior (about 85% of the stream) with a small mix in of teacher baked real public OpenML tables (about 15%), and evaluated zero shot on held out real OpenML datasets that are excluded from training by dataset id and name. No customer data and no benchmarked test set is in the training stream. The synthetic prior is in prior.py and the training objective is in train.py, which reproduces the method (python train.py --smoke).

Intended use, limitations, and safety

What it is for: fast, zero shot predictions and autofill on small to medium tables where you do not want to set up and tune a model: spreadsheet autofill, in database prediction, quick what if estimates, and calibrated triage where the confidence matters as much as the answer. It is a good first pass and a strong no tuning baseline.

What it is not: it is not a replacement for a tuned gradient boosted tree when you want maximum accuracy, especially on regression, where it is behind trees and ships error bars instead of headline accuracy. It is capped at 100 features and 10 classes. It is trained and validated on standard public tabular benchmarks; generalization to a very different distribution should be checked on your own held out data before you rely on it. Do not use it as the sole basis for a consequential decision.

How to cite

@misc{lowdownlabs_felatab,
  title  = {FelaTab: an in context tabular foundation model on the FELA linear attention base},
  author = {Lowdown Labs},
  year   = {2026},
  note   = {Model card}
}

Model family

This is part of the FELA family from Lowdown Labs: one linear attention architecture across many modalities, all CPU native and subquadratic. Sibling repos share no weights, so none carries a base_model link.

License

Apache-2.0. The model weights and the code are both under Apache-2.0 (see LICENSE): no separate model license and no non commercial restriction. Trained on synthetic data; the OpenML datasets are used only for evaluation, not training. We distribute weights only.