Downloads · 30 days
13
15% of all-time downloads
jensjepsen/danish-lm-400m-sft-v30-avg-top7
danish-lm-400m-sft-v30-avg-top7 is a machine learning model from jensjepsen. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Danish 400M SFT model, 2048-token context, uniform average of the top-7 checkpoints from the v30 training run at 4 epochs (3 initial + 1 continuation from checkpoint-49938, same wandb run 41u9zx6o).
Downloads · 30 days
13
15% of all-time downloads
All-time downloads
85
Public
Parameters
415M
15.9 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors14.9 GB · 94%
From the Hugging Face model README
Danish 400M SFT model, 2048-token context, uniform average of the top-7
checkpoints from the v30 training run at 4 epochs (3 initial + 1
continuation from checkpoint-49938, same wandb run 41u9zx6o).
Selected by aggregate of 9 downstream metrics (gsm8k, sciq, cit-gen, cit-mc, piqa, arc, gpqa, textman_summary, textman_rewrite).
jensjepsen/danish-lm-400m-base-ropext2048-v1jensjepsen/danish-tokenizerMix (19 SFT sources, ~2.24M packed training rows):
textman, rc, reason, metamath,
algebra, arith-chain, wiki-grounded, wiki-closedqa, wiki-mc-letters,
wp-v2, sciq, gsm8k, if-v4, text-to-question-v2, wiki-closedqa-stem-v1,
wiki-broadqa-stem-v1)danish-sci-reasoning-v1, danish-sci-factcheck-v1,
danish-sci-taskgen-v1 (~360k rows total, 16% of mix)Hyperparameters: 4 epochs, bs=128, ga=1, lr=3e-5 constant + 500 warmup, adamw_bnb_8bit, flatten-packing (FA2 varlen), fp32 weights + bf16 compute. ~7.6h total on 1×H100 (6h initial + 1.6h continuation).
Selected by callback aggregate (9 metrics, mean):
| step | epoch | agg |
|---|---|---|
| 54093 | 3.25 | 0.330 |
| 37449 | 2.25 | 0.328 |
| 66576 | 4.00 | 0.327 |
| 62415 | 3.75 | 0.324 |
| 33288 | 2.00 | 0.323 |
| 41610 | 2.50 | 0.320 |
| 45771 | 2.75 | 0.320 |
Available in constituents/ (model files only, no optimizer).
| eval | v29-avg7 | v30-avg7 (ep3) | v30-avg7 (ep4) | vs v29 |
|---|---|---|---|---|
| GSM8K[da] gen | 17.31 | 16.48 | 17.54 | ✓ +0.23 |
| SciQ[da] gen | 60.30 | 60.00 | 62.00 | ✓ +1.70 |
| Cit-gen | 29.9 | 29.20 | 27.50 | ✗ −2.4 |
| Cit-MC | 49.3 | 48.20 | 48.80 | ✗ −0.5 |
| mean(4) | 39.20 | 38.47 | 38.96 | −0.24 |
| IFEval-DA prompt-strict | 23.1 | 22.5 | 23.9 | ✓ +0.8 |
| IFEval-DA inst-strict | 36.6 | 36.1 | 37.2 | ✓ +0.6 |
| Textman summary (ChrF++) | 45.01 | 45.78 | 45.35 | ✓ +0.34 |
| Textman rewrite (ChrF++) | 48.62 | 49.34 | 48.49 | ✗ −0.13 |
| SciQ MC | — | 60.10 | 61.20 | new |
| PIQA[da] | — | 56.0 | 56.0 | new |
| ARC[da] | — | 26.82 | 27.76 | new |
| GPQA-Diamond[da] | — | 21.72 | 21.72 | new |
Verdict: Beats v29 on IFEval / GSM8K / SciQ / textman_summary, matches elsewhere. Only clear loss vs v29 is cit-gen (−2.4pp). Extra epoch closed the v30-ep3 gap: mean(4) went from −0.73 (ep3) to −0.24 (ep4).
When to pick which:
/ — averaged model (production; load with AutoModelForCausalLM.from_pretrained)checkpoint-66584/ — full training state (model + optimizer.pt + scheduler.pt +
rng_state.pth + trainer_state.json + training_args.bin), resumable via HF Trainer
--resume-from-checkpoint. Represents the ep 4.0 final training step.constituents/step-<N>-agg-<A>/ — the 7 individual best ckpts averaged into
root (model files only, no optim). Useful for re-averaging with different
weights or debugging drift.jensjepsen/danish-lm-400m-sft-v30-ep3-avg-top7 — same run at ep 3.0jensjepsen/danish-lm-400m-sft-v29-avg-top7 — v29 mix baseline (16 sources)jensjepsen/danish-lm-400m-base-ropext2048-v1 — base modelThe STEM triple (sci-reasoning / factcheck / taskgen) added new metric coverage (PIQA/ARC/GPQA measurable) but per-dataset investigation showed mixed quality — worked_calc trains correct arithmetic but 400M capacity can't leverage it; mechanism has occasional term-confusion; factcheck closed-view responses trained a subtle "Artiklen…" citation habit that bleeds into freeform generation. See notes in the parent v30 project for v31 remediation direction.