Downloads · 30 days
9
14% of all-time downloads
jensjepsen/danish-lm-400m-sft-v29-avg-top7
danish-lm-400m-sft-v29-avg-top7 is a machine learning model from jensjepsen. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Danish 400M-parameter SFT model with 2048-token context, uniform-average of the top-7 checkpoints from the v29 training run (by mean of 4 downstream evals: gsm8k, sciq, cit-gen, cit-mc).
Downloads · 30 days
9
14% of all-time downloads
All-time downloads
63
Public
Parameters
415M
1.7 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors1.7 GB · 100%
From the Hugging Face model README
Danish 400M-parameter SFT model with 2048-token context, uniform-average of the top-7 checkpoints from the v29 training run (by mean of 4 downstream evals: gsm8k, sciq, cit-gen, cit-mc).
jensjepsen/danish-lm-400m-base-ropext2048-v1jensjepsen/danish-tokenizervs v28's mix, added 3 task-expansion datasets (RC / reasoning / text- manipulation) grounded in Danish Wikipedia:
jensjepsen/danish-rc-v1 — 5 subtypes, ~80k rowsjensjepsen/danish-reason-v1 — 6 subtypes, ~120k rowsjensjepsen/danish-textman-v1 — 6 subtypes, ~120k rowsPlus v28's original 13 sources (v25 mix). 16 SFT sources total, ~1.8M packed training rows, 3 epochs, bs=128, ga=1, lr=3e-5 constant + 500 warmup, adamw_bnb_8bit, flatten-packing (FA2 varlen), fp32 weights + bf16 compute. ~4.5h on 1×H100.
| step | epoch | agg |
|---|---|---|
| 21180 | 1.50 | 0.271 |
| 38830 | 2.75 | 0.252 |
| 28240 | 2.00 | 0.249 |
| 24710 | 1.75 | 0.246 |
| 31770 | 2.25 | 0.245 |
| 42360 | 3.00 | 0.241 |
| 35300 | 2.50 | 0.240 |
| eval | v29-avg7 | v29-avg3 | v28-avg3 | v25-avg3 |
|---|---|---|---|---|
| GSM8K[da] gen | 17.31 | 17.01 | 17.39 | 19.24 |
| SciQ[da] gen | 60.30 | 59.00 | 61.90 | 62.00 |
| Cit-gen | 29.9 | 30.3 | 29.40 | 30.80 |
| Cit-MC | 49.3 | 49.4 | 49.60 | 49.70 |
| mean | 39.20 | 38.93 | 39.57 | 40.44 |
| IFEval-DA prompt-strict | 23.1 | 23.1 | 24.1 | 22.7 |
| IFEval-DA inst-strict | 36.6 | 36.6 | 37.1 | 36.6 |
Marginally below v28-avg-top3 on standard benchmarks. However, qualitative probes on real Danish Wikipedia articles show v29 handles summarization, extraction (JSON), style-transfer, genre-transform (tweet/casual/etc.), and analogy whereas v28-avg-top3 produces empty output or loops on those same prompts. The task-expansion gains aren't visible in current downstream benchmarks (which don't measure text↔text transformations) but are real.