Downloads · 30 days
18
5% of all-time downloads
jensjepsen/danish-lm-400m-grpo-mixed-v1-best3-step3250
danish-lm-400m-grpo-mixed-v1-best3-step3250 is a text generation model from jensjepsen. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
Rank-3 best-of-3 snapshot from a GRPO run that interleaved GSM8K + IFEval-DA (combined our-46 + Google-schema constraints) as a per-example dispatched reward. Base: jensjepsen/danish-lm-400m-sft-v31-avg-top3.
Downloads · 30 days
18
5% of all-time downloads
All-time downloads
369
Public
Parameters
415M
1.7 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors1.7 GB · 100%
From the Hugging Face model README
Rank-3 best-of-3 snapshot from a GRPO run that interleaved GSM8K + IFEval-DA (combined our-46 + Google-schema constraints) as a per-example dispatched reward. Base: jensjepsen/danish-lm-400m-sft-v31-avg-top3.
reward_gsm8k on gsm8k rows, reward_ifeval_combined (our 46 constraints + Google IFEval schema) on IF rows.jensjepsen/danish-if-grpo-combined-v1 (10k) interleaved with jensjepsen/danish-gsm8k:sft:train (~7.5k), 50/50 mix.jensjepsen/danish-lm-400m-grpo-if-combined-v1-step2400 (combined-run peak) at step 2400 into an already-in-progress mixed run; current model is at step 3250 of that mixed schedule.ifeval-da prompt-strict + inst-strict + gsm8k pass@1 (each ∈ [0, 1]). Best-K tracker in scripts/train_grpo_verifier.py (top-3 rolling, model+tokenizer only, no optimizer).Baseline column is jensjepsen/danish-lm-400m-sft-v31-avg-top3 (from its README).
danish-foundation-models/ifeval-da, n=539)Uses our patched Google verifier chain — DFM renamed change_case:capital_letters / change_case:lowercase_letters / detectable_format:constrained_response_with_argument without publishing verifier code; patches document divergence.
| metric | base | step 3250 | Δ |
|---|---|---|---|
| prompt-strict | 21.2 | 29.3 | +8.1 |
| prompt-loose | 22.0 | 29.7 | +7.7 |
| inst-strict | 35.2 | 44.5 | +9.3 |
| inst-loose | 35.8 | 44.9 | +9.1 |
| eval | base | step 3250 | Δ |
|---|---|---|---|
| GSM8K 0-shot pass@1 (n=1317) | 17.39 | 24.60 | +7.21 |
| SciQ open-Q pass@1 (n=1000) | 13.50 | 13.80 | +0.30 |
| CIT-gen (n=720) | 29.86 | 29.2 | -0.66 |
| textman_summary chrF++ | 41.11 | 40.94 | -0.17 |
| textman_rewrite chrF++ | 46.51 | 47.20 | +0.69 |
| eval | base | step 3250 | Δ |
|---|---|---|---|
| CITMC (n=720) | 48.19 | 48.75 | +0.56 |
| SciQ-MC (n=1000) | — | 59.6 | — |
| PIQA (n=100 full test) | 53 (n=50) | 59.0 | +6 |
ARC-Easy chat-MC (jensjepsen/danish-arc:arc_easy:test) | 44.40 | 42.55 | -1.85 |
ARC-Challenge chat-MC (jensjepsen/danish-arc:arc_challenge:test) | 29.35 | 28.33 | -1.02 |
| eval | base | step 3250 | Δ |
|---|---|---|---|
| ARC-Easy logprob (n=2376) | 40.61 | 40.45 | -0.16 |
| ARC-Challenge logprob (n=1172) | 27.47 | 27.47 | 0.00 |
from transformers import AutoModelForCausalLM, AutoTokenizer
m = AutoModelForCausalLM.from_pretrained("jensjepsen/danish-lm-400m-grpo-mixed-v1-best3-step3250")
t = AutoTokenizer.from_pretrained("jensjepsen/danish-lm-400m-grpo-mixed-v1-best3-step3250")
Chat format (single-turn): <|user|>{q}<|end|><|assistant|>, stop on <|end|>.
best1-step3375, best2-step4375Weights only (no optimizer/scheduler) — for eval/inference.