house-prices-tabular-models

House Prices - Tabular Models (8 architectures + interpretability)

Pre-trained models for the t22000t/house-prices-tabular dataset, covering all eight architectures from the tabular-data-modelling-pipeline: two gradient-boosted machines (CatBoost, XGBoost) and six deep-learning models (CANN, CANN-GBM, FT-Transformer, TabM, LocalGLMnet, DRN).

v3 release - adds the full interpretability stack: SHAP TreeExplainer for the GBMs, Captum Integrated Gradients for the DL models, per-layer attention for FT-Transformer, residual analysis for CANN / CANN-GBM, per-row coefficients for LocalGLMnet, and full distributional outputs (mean, variance, quantiles, VaR) for DRN. See INTERPRETABILITY.md and dashboard_dl_interpretability.html.

Results

All 8 architectures trained with default hyperparameters (no Optuna tuning), 3-seed ensembles for the DL models, gamma family + log link.

• Test set: 304 rows (20% of 1,460)
• Target: SalePrice (USD)
• Loss: Gamma deviance via reg:gamma (XGBoost) / Tweedie:variance_power=1.99 (CatBoost) / explicit gamma NLL (DL)
• Cap: 99.5th percentile (= $555,355; 6 rows winsorised)
• Random seed: 42

Why FT-Transformer and TabM underfit

With only ~1,150 training rows, transformer-class models have nothing like enough data to learn meaningful attention patterns. They converge to near-constant predictions (MAE ≈ mean target). This is the expected behaviour for these architectures on small tabular datasets - the literature consistently shows they need 10k+ rows to outperform GBMs.

For a comparison where these architectures actually compete, see the companion drop on Bike Sharing (17k rows): t22000t/bike-sharing-tabular-models.

Interpretability (new in v3)

Multiple methods applied side by side so findings can be triangulated:

Cross-method consensus (the high-confidence finding)

Both CatBoost and XGBoost ranked the following four features in their top-5 most important for predicting SalePrice. Cross-method agreement is a strong signal - independent methods identifying the same drivers means the finding is unlikely to be a method-specific artefact:

Note the derived feature TotalSF (defined in configs/example_house_prices.py as TotalBsmtSF + 1stFlrSF + 2ndFlrSF) is more predictive than any of its components individually - the pipeline's derived_features mechanism paid off here.

Full breakdown - per-method top-10 tables, LocalGLMnet coefficient distributions, sign-stability analysis - is in INTERPRETABILITY.md. For the interactive plots see dashboard_dl_interpretability.html.

Files

Loading and inference

CatBoost (single best non-XGB baseline)

from huggingface_hub import hf_hub_download
from catboost import CatBoostRegressor
import pandas as pd

path = hf_hub_download("t22000t/house-prices-tabular-models", "catboost.cbm")
model = CatBoostRegressor()
model.load_model(path)

df = pd.read_csv("hf://datasets/t22000t/house-prices-tabular/train.csv")
features = [  # see configs/example_house_prices.py for the full list
    "LotArea", "YearBuilt", "YearRemodAdd", "TotalBsmtSF", "1stFlrSF",
    "2ndFlrSF", "GrLivArea", "FullBath", "BedroomAbvGr", "TotRmsAbvGrd",
    "GarageCars", "GarageArea", "OverallQual", "OverallCond",
    "MSZoning", "Street", "LotShape", "Neighborhood", "BldgType",
    "HouseStyle", "RoofStyle", "ExterQual", "Foundation", "Heating",
    "CentralAir", "KitchenQual", "SaleType", "SaleCondition",
]
preds = model.predict(df[features])

XGBoost (best overall on this dataset)

from huggingface_hub import hf_hub_download
import xgboost as xgb

path = hf_hub_download("t22000t/house-prices-tabular-models", "xgboost.json")
booster = xgb.Booster()
booster.load_model(path)

XGBoost predictions need the exact feature order used at training time. Clone the pipeline repo to reproduce the preprocessing path.

Deep-learning models (CANN, CANN-GBM, FT-T, TabM, LocalGLMnet, DRN)

Each architecture ships as a 3-seed ensemble of PyTorch state-dicts. Reconstructing them requires the matching architecture class from the pipeline package. Easiest path:

git clone https://github.com/timothy22000/tabular_data_modelling_pipeline
cd tabular_data_modelling_pipeline
pip install -e ".[all]"

Then load each ensemble member's state dict into the right module (see modelling/models/cann.py etc).

Training configuration

To reproduce:

git clone https://github.com/timothy22000/tabular_data_modelling_pipeline
cd tabular_data_modelling_pipeline
pip install -e ".[all]"
python scripts/download_data.py --dataset house_prices

OMP_NUM_THREADS=1 python train.py \
    --config configs/example_house_prices.py \
    --input data/house_prices.csv \
    --skip-tuning --skip-interpretability \
    --architectures catboost xgboost cann cann_gbm ft_transformer tabm localglmnet drn

(OMP_NUM_THREADS=1 is only needed on macOS arm64; Linux is unaffected.)

Limitations

• No Optuna tuning. Defaults only. With tuning, expect ~0.02-0.04 Gini lift across the board.
• FT-Transformer & TabM underfit. Dataset is too small (~1.2k training rows) - see the companion Bike Sharing drop for a fair comparison.
• No interpretability artefacts. Captum attributions and partial-dependence plots were skipped to keep wall-clock manageable. The pipeline supports them - drop --skip-interpretability to compute on re-run.
• Random split, not stratified. SalePrice is right-tailed; a quantile-stratified split would give a more representative test set.
• Trained on Kaggle's train.csv only (test.csv is unlabelled). Not directly comparable to the Kaggle leaderboard.

Intended use

• Baseline for tabular DL research on small-N regression problems.
• Teaching end-to-end tabular pricing pipelines (8 architectures with one CLI).
• Sanity check for reimplementations of any of the 8 architectures on gamma-family data.

Citation

@software{tabular_data_modelling_pipeline,
  author = {Mun, Timothy},
  title  = {tabular-data-modelling-pipeline},
  url    = {https://github.com/timothy22000/tabular_data_modelling_pipeline},
  year   = {2026}
}

@article{decock2011ames,
  author  = {De Cock, Dean},
  title   = {Ames, Iowa: Alternative to the Boston Housing Data},
  journal = {Journal of Statistics Education},
  volume  = {19},
  number  = {3},
  year    = {2011}
}

Please also cite the individual architecture papers - see the main repo README for the full reference list.

License

MIT for the model code and pipeline. Underlying dataset under Kaggle competition terms (free use with attribution).

Related

• 📂 Dataset: t22000t/house-prices-tabular
• 🤖 Companion: t22000t/bike-sharing-tabular-models - Poisson family, 17k rows
• 📦 Pipeline: tabular-data-modelling-pipeline