Downloads · 30 days
0
212M-FineWeb-Experiment
hudsongouge/212M-FineWeb-Experiment
212M-FineWeb-Experiment is a machine learning model from hudsongouge. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
This repo is not a single model. It contains multiple ~212M-parameter checkpoints from one FineWeb-Edu experiment line.
Access
Public
Updated Jul 9, 2026
Repo size
3.9 GB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.pt3.9 GB · 100%
At a glance
- License
- mit
- Access
- Public
- Created
- Jul 6, 2026
- Updated
- Jul 9, 2026
- SHA
- c23c4ad0
Model card
From the Hugging Face model README
212M FineWeb Experiment
This repo is not a single model. It contains multiple ~212M-parameter checkpoints from one FineWeb-Edu experiment line.
Base pretrain data: FineWeb-Edu (~3.8B tokens).
Size (all variants): ~212M parameters (14 layers, d_model=768, 24k vocab).
Models in this repo
| Folder | What it is |
|---|---|
01-pretrain-fineweb-edu-tst | Base pretrained on FineWeb-Edu |
02-sft-curriculum | Chat SFT warm-start on top of the base |
03-compare-ours | Short reasoning-comparison run (Ours) |
04-compare-grpo | Short reasoning-comparison run (GRPO) |
05-compare-star | Short reasoning-comparison run (STaR) |
06-compare-dpo | Short reasoning-comparison run (DPO) |
07-prod-ours-2h | Longer Ours run (seq 2048) |
08-prod-ours-latent-2h | Longer latent-Ours run (seq 2048) |
09-prod-ours-latent-v4 | Latent-Ours v4 run (seq 2048) |
Each variant lives under variants/<folder>/ with its own config (and checkpoint when uploaded).
- License
- mit
- Languages
- en
- Created
- Jul 6, 2026
- Updated
- Jul 9, 2026