Downloads · 30 days
15
1% of all-time downloads
amosluna/babylm-2026-strict-small-mdlm-seed42
babylm-2026-strict-small-mdlm-seed42 is a machine learning model from amosluna. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
BabyLM 2026 Strict-Small (English) submission checkpoint. A LLaDA/MDLM-style absorbing-state masked-diffusion language model on a GPT-2-scale bidirectional Transformer, trained on <=10M unique words for <=10 epochs (<…
Downloads · 30 days
15
1% of all-time downloads
All-time downloads
1.2K
Public
Parameters
98.4M
7.5 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors394 MB · 100%
From the Hugging Face model README
MD_base seed 42BabyLM 2026 Strict-Small (English) submission checkpoint. A LLaDA/MDLM-style absorbing-state masked-diffusion language model on a GPT-2-scale bidirectional Transformer, trained on <=10M unique words for <=10 epochs (<=100M words seen).
MD_base (masked diffusion)429999673719 branches chck_1M ... chck_100Mfrom transformers import AutoModelForMaskedLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("amosluna/babylm-2026-strict-small-mdlm-seed42")
model = AutoModelForMaskedLM.from_pretrained("amosluna/babylm-2026-strict-small-mdlm-seed42", trust_remote_code=True)
# Intermediate checkpoint:
# model = AutoModelForMaskedLM.from_pretrained("amosluna/babylm-2026-strict-small-mdlm-seed42", revision="chck_10M", trust_remote_code=True)
A masked-diffusion denoiser is scored like a masked LM (per-token
pseudo-log-likelihood), so the official BabyLM pipeline evaluates it with the
mlm backend:
cd strict
./eval_zero_shot.sh amosluna/babylm-2026-strict-small-mdlm-seed42 mlm
./eval_finetuning.sh --model_path amosluna/babylm-2026-strict-small-mdlm-seed42 --seed 42
bash scripts/collate_preds.sh amosluna/babylm-2026-strict-small-mdlm-seed42 mlm strict-small --fast
See the project's docs/EVALUATION.md for the diffusion-native ELBO scorer and
the inference-time layer-duplication variant.