Downloads · 30 days
0
0% of all-time downloads
jensjepsen/danish-lm-400m-grpo-mixed3-combinedv4-lr2.5e-6
danish-lm-400m-grpo-mixed3-combinedv4-lr2.5e-6 is a machine learning model from jensjepsen. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
Top-3 checkpoints (by tracked best-k composite) from a 3-way GRPO run (instruction-following + gsm8k + JSON schema, 1:1:1) on jensjepsen/danish-lm-400m-sft-v31-avg-top3.
Downloads · 30 days
0
0% of all-time downloads
All-time downloads
31
Public
Repo size
5 GB
Likes
0
Public
Click a slice to open those files.
.safetensors5 GB · 100%
From the Hugging Face model README
Top-3 checkpoints (by tracked best-k composite) from a 3-way GRPO run (instruction-following + gsm8k + JSON schema, 1:1:1) on jensjepsen/danish-lm-400m-sft-v31-avg-top3.
Run paused at step 22458 / 89709 (~25%, 8h23m wall).
Subfolders, best-k ranked:
| subfolder | best-k score |
|---|---|
step-20125 | 2.414 |
step-20000 | 2.401 |
step-18875 | 2.400 |
jensjepsen/danish-lm-400m-sft-v31-avg-top3jensjepsen/danish-if-grpo-combined-v4 (10k rows)--gsm-frac 0.333 --json-frac 0.333dr_grpoGRPO_DAPO_RESAMPLE=1 GRPO_DAPO_FRESH_PROMPTS=1 GRPO_DAPO_FRESH_MATCH_TASK=1)--vllm-gpu-memory-utilization 0.40Project danish-lm-grpo, run grpo_mixed3_maxtok768_ngen32_v31base_drgrpo_dapofresh_combinedv4_lr2.5e-6_kl0.01
0-shot, greedy (do_sample=False, num_beams=1, repetition_penalty=1.1),
chat format, full test splits. All three suites: 13/13 evals exited rc=0.
Base column is reproduced from the
v31 SFT model card
(avg-top3), not re-measured here. — = not reported there.
| eval | metric | step-20125 | step-20000 | step-18875 | v31 SFT base |
|---|---|---|---|---|---|
| ifeval-da | prompt-strict | 44.3 | 44.2 | 45.8 | 21.2 |
| prompt-loose | 46.2 | 44.9 | 47.7 | 22.0 | |
| inst-strict | 60.1 | 60.1 | 61.0 | 35.2 | |
| inst-loose | 61.4 | 61.1 | 62.1 | 35.8 | |
| ifbench-da | prompt-strict | 10.7 | 9.7 | 9.3 | — |
| prompt-loose | 14.7 | 14.3 | 13.3 | — | |
| inst-strict | 11.9 | 10.5 | 11.0 | — | |
| inst-loose | 16.9 | 16.3 | 15.7 | — | |
| gsm8k | pass@1 | 28.47 | 27.18 | 27.64 | 17.39 |
| json | mean_reward | 1.072 | 1.070 | 1.065 | — |
| sciq-gen | pass@1 | 13.10 | 13.70 | 13.50 | 13.50 |
| sciq-mc | acc | 55.50 | 55.40 | 57.00 | — |
| cit-gen | acc | 26.2 | 26.7 | 26.7 | 29.86 |
| cit-mc | acc | 45.0 | 44.6 | 48.3 | 48.19 |
| arc-easy | chat-MC acc | 38.05 | 38.38 | 38.51 | 44.40 |
| arc-challenge | chat-MC acc | 26.02 | 25.68 | 27.13 | 29.35 |
| openbookqa | chat-MC acc | 35.80 | 35.60 | 36.60 | 35.40 |
| piqa | chat-MC acc | 51.00 | 51.00 | 52.00 | 53.00 |
| textman-summary | ChrF++ | 39.93 | 40.09 | 39.88 | 41.11 |
| textman-rewrite | ChrF++ | 47.00 | 46.25 | 47.07 | 46.51 |
ifbench-da is scored with a Danish-tolerant variant of the AllenAI IFBench verifiers (Danish conjunctions, pronouns, alphabet, stopwords, verb lexicon, syllable counting, and answer options accepted alongside the English forms).
MC parse failures:
| ckpt | arc-easy | arc-challenge | openbookqa |
|---|---|---|---|
| step-20125 | 96 / 2376 | 22 / 1172 | 0 / 500 |
| step-20000 | 96 / 2376 | 22 / 1172 | 0 / 500 |
| step-18875 | 97 / 2376 | 22 / 1172 | 0 / 500 |
<|user|>{prompt}<|end|><|assistant|>{answer}<|end|>