Downloads · 30 days
704
25% of all-time downloads
jensjepsen/danish-lm-400m-icl-v3
danish-lm-400m-icl-v3 is a text generation model from jensjepsen. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
400M Danish model tuned for in-context schema and format induction: given a few worked examples in a single user turn and no instruction, infer both the output schema and the output format from the examples and apply…
Downloads · 30 days
704
25% of all-time downloads
All-time downloads
2.8K
Public
Parameters
415M
2.6 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors1.7 GB · 63%
From the Hugging Face model README
400M Danish model tuned for in-context schema and format induction: given a few worked examples in a single user turn and no instruction, infer both the output schema and the output format from the examples and apply them to a new passage.
jensjepsen/danish-lm-400m-sft-v31-avg-top3jensjepsen/danish-icl-schema-format-v3 sft:train, 33,933 rows, ICL-only, no ballastadamw_bnb_8bit, seq 3072, FA2 varlen (DataCollatorWithFlattening)Exact match on the parsed object, greedy, n=400 per split. Base = the v31 checkpoint this was tuned from, measured in the same session.
| split | base | this model |
|---|---|---|
| val (seen schemas + formats, held-out passages) | 0.2% | 77.8% |
| eval_schema (unseen schemas) | 0.0% | 41.5% |
| eval_format (unseen formats) | 0.0% | 25.8% |
| eval_both | 0.0% | 15.5% |
Trained formats on unseen schemas are even: numbered 51.7, tsv 43.8, tagged 43.6, kv_colon 41.0, json 38.7, kv_bracket 36.5, kv_arrow 35.5.
Meaning-free keys are about as easy as real Danish field names (val 76.7 vs 78.7; eval_schema 40.1 vs 42.9), which is the evidence the mapping is read from the demonstrations rather than from field-name semantics — the base scored 0.6% on symbol rows.
Format transfer is delimiter substitution, not structural generalisation.
On unseen formats, kv_eq (k=v, one delimiter away from trained formats)
reaches 78.5%, while bracket_pair ([k]v[/k]) scores 0.0% and
brace_pair ({k}v{/k}) 0.7% — despite bracket_pair being structurally
identical to the trained tagged (<k>v</k>). The model copies the opening
delimiter ~100% of the time and gets every value right ~60% of the time, but
produces the matched close only ~37%, falling back to the trained </tag>
form. To get a format into the repertoire, train on it.
Out-of-distribution tasks work, limited by task ability not format. On
dane_plus NER (never trained on; 3-shot, no instruction) it parses 100% in
tagged and kv_colon against the base's 52%/24%, reaching 26-31 entity F1
where the base gets 0-2.4. Residual errors are entity typing, not format.
Trained ICL-only. The predecessor trained on a single format became rigid about structured output; multi-format training was the fix. General benchmarks were roughly flat but drifted down over the run (gsm8k 17.0 to 12.5 at n=200, citgen 28.5 to 24.0) — mix in ballast if that matters for your use.
Exemplars go inside one user turn. This model family breaks on multi-turn few-shot (the v31 card records GSM8K 18.7 to 2.1%).
<|user|>Eksempler:
Tekst:
<passage>
Svar: <rendered answer>
Tekst:
<passage>
Svar: <rendered answer>
Tekst:
<target passage>
Svar:<|end|><|assistant|>