Downloads · 30 days
8
20% of all-time downloads
jensjepsen/danish-lm-400m-sft-v30-ep3-avg-top7
danish-lm-400m-sft-v30-ep3-avg-top7 is a machine learning model from jensjepsen. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Danish 400M SFT model, 2048-token context, uniform average of the top-7 checkpoints from v30 training at 3 epochs (ep3 marker distinguishes this from the planned post-continuation v30 avg).
Downloads · 30 days
8
20% of all-time downloads
All-time downloads
40
Public
Parameters
415M
1.7 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors1.7 GB · 100%
From the Hugging Face model README
Danish 400M SFT model, 2048-token context, uniform average of the top-7
checkpoints from v30 training at 3 epochs (ep3 marker distinguishes
this from the planned post-continuation v30 avg).
Selected by aggregate of 9 downstream metrics (gsm8k, sciq, cit-gen, cit-mc, piqa, arc, gpqa, textman_summary, textman_rewrite).
jensjepsen/danish-lm-400m-base-ropext2048-v1jensjepsen/danish-tokenizervs v29's 16-source mix, added 3 STEM-derived SFT sources (~360k rows) built by walking the STEM category tree of da_wiki (4986 pages under Fysik / Kemi / Biologi / Matematik / Astronomi / Geologi):
jensjepsen/danish-sci-reasoning-v1 — worked-calc / mechanism /
counterfactual open+closed reasoning chains (~88k rows)jensjepsen/danish-sci-factcheck-v1 — balanced SAND/FALSK claim
verification (~87k rows)jensjepsen/danish-sci-taskgen-v1 — reverse-direction: model
generates tasks from an article, single/multi/qonly/balanced-batch
variants (~186k rows)19 SFT sources total, ~2.24M packed training rows, 3 epochs, bs=128, ga=1, lr=3e-5 constant + 500 warmup, adamw_bnb_8bit, flatten-packing (FA2 varlen), fp32 weights + bf16 compute. ~6h on 1×H100.
| step | epoch | agg |
|---|---|---|
| 37449 | 2.25 | 0.328 |
| 33288 | 2.00 | 0.323 |
| 41610 | 2.50 | 0.320 |
| 45771 | 2.75 | 0.320 |
| 49932 | 3.00 | 0.320 |
| 29127 | 1.75 | 0.318 |
| 24966 | 1.50 | 0.316 |
| eval | v30-ep3-avg7 | v29-avg7 | v28-avg3 | v25-avg3 |
|---|---|---|---|---|
| GSM8K[da] gen | 16.48 | 17.31 | 17.39 | 19.24 |
| SciQ[da] gen | 60.00 | 60.30 | 61.90 | 62.00 |
| Cit-gen | 29.2 | 29.9 | 29.40 | 30.80 |
| Cit-MC | 48.2 | 49.3 | 49.60 | 49.70 |
| mean(4) | 38.47 | 39.20 | 39.57 | 40.44 |
| IFEval-DA prompt-strict | 22.5 | 23.1 | 24.1 | 22.7 |
| IFEval-DA inst-strict | 36.1 | 36.6 | 37.1 | 36.6 |
| eval | v30-ep3-avg7 | v29-avg7 | notes |
|---|---|---|---|
| PIQA[da] | 56.0 | — | 2-choice physical commonsense |
| ARC[da] | 26.82 | — | 4-5 choice science MC |
| GPQA-Diamond[da] | 21.72 | — | 4-choice STEM, below random 25% (parsefails 12/198) |
| Textman summary (ChrF++) | 45.78 | 45.01 | reference-based (val split, n=1000) |
| Textman rewrite (ChrF++) | 49.34 | 48.62 | reference-based (val split, n=999) |
Regression on 4 legacy metrics (−0.73pp mean, driven by cit-MC −1.1 and gsm8k −0.83), and −0.5pp on IFEval. Cause: adding 3 STEM sources (~16% of mix) diluted training signal on the civics/factoid-heavy sources (wiki-grounded, wiki-closedqa, text-to-question).
Gain: new domain coverage for STEM reasoning, physical commonsense (PIQA), and structured MC (ARC). Averaging picked up +1pp on textman_summary and +1.7pp on textman_rewrite over the best single ckpt, confirming late-training instability that avg smooths over.
Use v30 when STEM/reasoning breadth matters; v29-avg7 remains the stronger pick for pure civics/knowledge tasks.