tsfm-toto2-4m-100k-behavior-probe

TSFM tokenizer ablation — Toto-2-4m-recipe clone (v10)

VQ-VAE OPTIMIZATION ablation (codebook health + next-code objective vs raw patches) on a ~3.6M-param decoder-only patched transformer trained on the OFFICIAL TempoPFN synthetic prior generators (cloned from automl/TempoPFN; GP/KernelSynth families capped at 6,144 steps, so the long pool uses the cheap families only) with contiguous patch masking, a 9-level quantile head, NorMuon+AdamW, and index RoPE at time-scaled positions. Horizon decoding is fixed patch-32 in every arm; only the CONTEXT tokenizer varies:

Each subfolder is one (arm, seed) run: model.pt (final), ckpt_15000.pt (rank-stability snapshot), config.json, results.json (dev-GIFT CRPS + long-horizon probe). Dev metrics use a fixed 14-task GIFT-Eval subset — NOT the full leaderboard; treat numbers as ablation-internal, not comparable to published GIFT scores. Generated by the v10 experiment notebook.

Results

=== transfer curve: GM-CRPS by checkpoint (dev-14, mean over seeds) ===
                    gm@12500  gm@25000  gm@37500  gm@50000  gm@62500  gm@75000  gm@87500  gm@final  gm_final_std  long_season  params_m      perp  probe_amp  n
arm                                                                                                                                                            
Q0_n3_nat100k         0.2333    0.1937    0.1953    0.1961    0.1892    0.1948    0.1850    0.1854           NaN       0.2503     3.878  648.0925      0.231  1
Q1_n3_tpfnfull100k    0.2091    0.2170    0.2040    0.2374    0.2424    0.2183    0.2124    0.2103           NaN       0.2681     3.878  408.1310      0.382  1

--- tripwire checks ---
  Q0_n3_nat100k vs Q1_n3_tpfnfull100k: final gap=0.0249 -> sigma unknown (single seed)
  Q0_n3_nat100k: TRANSFER INVERSION — best at ckpt #7 (0.1850) vs final (0.1854); synthetic overfit persists
  Q1_n3_tpfnfull100k: TRANSFER INVERSION — best at ckpt #3 (0.2040) vs final (0.2103); synthetic overfit persists