satzklar-model

Model Card — satzklar-model (German UD biaffine dependency parser)

Model details

• Task: German dependency parsing (Universal Dependencies). Predicts, per token, the head index and the dependency relation label.
• Architecture: Dozat–Manning biaffine dependency parser (supar BiaffineDependencyParser).
• Encoder: deepset/gbert-base (German BERT, ~110 M params), fine-tuned.
• Head: biaffine arc scorer (arc_attn) + biaffine relation scorer (rel_attn); 49 relation labels (union of HDT + GSD).
• Output: head + relation only. No POS/morphology/case (the model is trained without those input features); obtain those from a separate component.
• License: CC BY-SA 4.0.

Intended use

• German sentence → dependency tree, as the attachment component of a grammar-analysis pipeline (e.g. deriving constituent/component trees, agreement checks, teaching feedback).
• Designed for cheap, deterministic CPU serving (int8, ~13.5 ms/sentence on 4 threads) inside a bounded-expert cluster (returns a parse + citation; abstains on non-parseable input).

Out of scope / cautions: the base model is trained on news/tech-news treebanks (HDT: heise.de 1996–2001; GSD: news/web); it is not tuned to learner German or the satzklar domain — that conditioning is pending. Tokenisation is whitespace + punctuation-peeling in the reference spoke; a proper German/MWT tokenizer will change results on contractions (im/am/zum/…).

Evaluation

Held-out test sets, UD-standard scoring (punctuation excluded, projective decoding off):

int8-quantized serving preserves accuracy (HDT-test UAS ≈ 0.978, LAS ≈ 0.968 on a 1,000-sentence subset). HDT figures are on par with reported biaffine results for this treebank (≈97.97 / 96.97); GSD is at the ceiling reported for that smaller, noisier treebank.

In-domain evaluation (satzklar production traffic)

Eval card for the deployment domain: real user sentences from the satzklar.net grammar app (short learner German, median 12 words), evaluated 2026-07-19 under a pre-registered protocol (PREREG_INDOMAIN_GOLD.md in the companion germandata repository — thresholds fixed before scoring, sample frozen before arbitration).

Gold construction. 800 sentences (seeded, stratified by length and parser-agreement status) drawn from 7,373 unique production sentences with zero overlap with HDT or GSD (all splits checked — the evaluation is uncontaminated by training data). Labels: Stanza-de ⋈ this model exact agreements as auto-silver (92% of tokens); disagreements resolved by a claude-sonnet-5 2-of-3 arbiter; residual three-way disagreements adjudicated per-token against written UD guidelines and GSD/HDT convention checks, with unresolvable tokens excluded. The gold set is model-consensus + treebank-grounded, not human-verified — treat in-domain numbers as [empirical] with a known silver bias toward both ensemble voices.

Bias-free companion numbers: on resolved disagreement tokens the model wins 65.1% (514/790) head-to-head against Stanza-de, and produces the exact full head-structure on 603/800 sentences (Stanza: 0.9640/0.9449 coarse-LAS, 492/800 full trees on the same gold). Worst-case bound (scoring every excluded token against this model): coarse-LAS ≥ 0.95.

Against the LLM it replaces: on the 234 GSD-test sentences where the app's production LLM parser (Claude Haiku 4.5) produced validated UD parses, this model scores UAS 89.9 / LAS 85.4 vs Haiku's 39.8 / 34.0 — same sentences, same gold, same scorer (results/haiku_overlap_234.json, reproducible via scripts/eval_haiku_overlap.py).

Why no in-domain fine-tuning: the registered decision rule (in-domain coarse-LAS within 2 points of GSD-test level AND disagreement win rate ≥ 0.50) passed with wide margin; residual in-domain errors concentrate in learner word-salad and convention-ambiguous constructions that additional silver training cannot address. The evaluation is distribution-conditional: it holds for current satzklar traffic and should be re-run if the usage distribution shifts.

Training

• Data: German-HDT ∪ German-GSD, 166,848 train / 19,233 dev sentences (UD CoNLL-U).
• Regime: 10 epochs, batch 1,000 tokens ×3 accumulation, lr 5e-5 (encoder) / ×20 (head), warmup 0.1; ~2 h wall on an RTX 5050 (8 GB), peak ~3.2 GB VRAM. Best dev at the final epoch.
• Fully reproducible from train_german_parser_full.py in this repository.

Data & licensing

German-HDT and German-GSD are distributed under CC BY-SA 4.0 (commercial use permitted; GSD dropped its earlier non-commercial restriction in 2019). The model is a derivative and is released under the same CC BY-SA 4.0 (share-alike + attribution). The future satzklar-conditioning data is first-party; it introduces no third-party licensing constraint on the released weights.