Downloads · 30 days
10
22% of all-time downloads
jensjepsen/danish-lm-400m-sft-v28-avg-top3
danish-lm-400m-sft-v28-avg-top3 is a machine learning model from jensjepsen. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Danish 400M-parameter SFT model with 2048-token context, uniform-average of the top-3 checkpoints from the v28 training run (by mean of 4 downstream evals: gsm8k, sciq, cit-gen, cit-mc).
Downloads · 30 days
10
22% of all-time downloads
All-time downloads
46
Public
Parameters
415M
1.7 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors1.7 GB · 100%
From the Hugging Face model README
Danish 400M-parameter SFT model with 2048-token context, uniform-average of the top-3 checkpoints from the v28 training run (by mean of 4 downstream evals: gsm8k, sciq, cit-gen, cit-mc).
jensjepsen/danish-lm-400m-base-ropext2048-v1 (RoPE-extended
pretrain, long_ppl 17.42 vs 748 for the 1024-ctx base)jensjepsen/danish-tokenizerDataCollatorWithFlattening (FA2 varlen, per-sample
RoPE)| step | epoch | agg |
|---|---|---|
| 23520 | 2.00 | 0.260 |
| 35280 | 3.00 | 0.257 |
| 32340 | 2.75 | 0.248 |
| eval | v28-avg-top3 | v27-avg-top3 (with tool-calls) | v25-avg-top3 (1024 base) |
|---|---|---|---|
| GSM8K[da] gen | 17.39 | 17.24 | 19.24 |
| SciQ[da] gen | 61.90 | 60.00 | 62.00 |
| Cit-gen | 29.40 | 28.50 | 30.80 |
| Cit-MC | 49.60 | 49.70 | 49.70 |
| mean | 39.57 | 38.86 | 40.44 |
v28 is the best 2048-context SFT model (v25 is stronger on downstream but capped at 1024 context).