bartholomew-experiments

BART Experiments

The complete training archive behind BART — 39 runs, their checkpoints, tokenizers, and evaluations, including every dead end.

📝 Read the write-up · 🌐 Unbounded Labs

If you want the model itself, use bart or bart-sft. This repository is for reproducing or inspecting how they were reached.

Layout

experiments/    39 runs — each owns its tokenizer and base checkpoints
evaluations/    vintage-core results across models
archive/        legacy paths (pre-lineage-v1)

Each base experiment owns its tokenizer and base checkpoints. SFT runs nest under their exact base parent (experiments/<base>/sft/<sft-run>/), and post-training runs nest under their exact SFT parent.

Reading experiment names

The prefix tells you the corpus generation:

The runs that matter

Think.Unbounded-d32 is the first d32 run, trained on midtrain mixtures whose ratios were computed by document count. Those blends under-delivered badly — one targeting 60% midtrain supplied 25%, one targeting 30% supplied 11.5%.

Think.Unbounded-d32-v2mix-cont is the repair: it branches from that run at step 5500, carries the optimizer state, and continues on corrected token-based mixtures. It is the model shipped as bart. The two configs side by side are the clearest record of that bug and its fix.

Its sft/ directory holds six fine-tuning variants. The one shipped as bart-sft is pre1930-curriculum-c3-robust-v2; the others — karpathy-modern-sft-v1, nanochat-default-datamatch-v1, c3-robust, and two c3-robust-v3 configs — are kept here.

clean1930s-d24-r12-ctx4096-sssl-fulltok-v1 is the d24 / 1.38B run; its tokenizer is the one every midtrain mixture was built against.

Note. Each config.json records repo and dataset names as they were at training time (jbduran/think.nano, think-dataset-clean-1930s, think-midtrain). Repo renames still resolve via Hub redirects, but the midtrain mixture subfolders they reference (mixed/v2/ratio_21/data) moved to mixtures/v2-by-tokens/ratio_21/data when that dataset was restructured. Configs are kept as historical records rather than rewritten.

Related

bart · bart-sft · bart-dataset-v3 · bart-midtrain · bart-dataset-scripts · bart-midtrain-scripts

License

MIT

---

Built by Unbounded Labs.