klitzeklein-europarl-translator

klitzeklein-europarl-translator — trained checkpoints

klitzeklein (German): teeny-tiny. Four transformer translators of 23–25M parameters, trained from scratch on Europarl on a single RTX 3060, from the research repo https://github.com/Marvel4U/klitzeklein-europarl-translator — a systematic small-scale comparison of specialization vs. general pretraining, encoder-decoder vs. decoder-only, language direction and architecture. Read the repo README for the full results, findings and limitations; this card documents the weights.

Checkpoints

Metrics: greedy decoding on 2000 held-out Europarl sentence pairs per direction (see repo). All models: 384-dim, 6 heads, pre-norm GELU blocks, joint BPE-4096 tokenizer with tied embeddings, dropout 0.2, bf16 training. Tokenizers are included in tokenizers/ (tokenizer_4096_m5.json for NL–EN, tokenizer_4096_pl_m5.json for PL–EN and EN–PL).

Format

Each model.pt is a plain torch.save dictionary: model_state_dict, model_config, extra (architecture, language direction, tokenizer path), training (final losses), schema_version. Optimizer states are stripped. The model classes live in the repo (plain_gpt_module/); the checkpoints are not 🤗 transformers models and do not load with from_pretrained.

Usage

git clone https://github.com/Marvel4U/klitzeklein-europarl-translator
cd klitzeklein-europarl-translator
pip install -r requirements.txt   # plus torch, see repo README
hf download mxv456/klitzeklein-europarl-translator --local-dir hf_models

# score a checkpoint on the held-out set (after preparing the data, see repo README)
python eval_translation.py hf_models/nl-en_encdec_9-3/model.pt --tasks translate --limit 0 --show-samples 5 --no-compile

# interactive translation with the decoder-only model
python translate.py -c hf_models/nl-en_decoder-only_12L/model.pt --src nl --tgt en -i

The tokenizer paths recorded in the checkpoints are relative to the repo root (bpe/tokenizers/...), which the cloned repo provides; the copies in tokenizers/ here are for reference.

Intended use and limitations

Research and teaching artifacts. Trained only on lowercased, diacritic-folded parliamentary proceedings (sentences ≤ 40 words); they do not generalise to everyday Dutch or Polish and are not competitive with production MT systems. Known artifact: the encoder-decoder models emit a spurious sentence-initial token ("and", "owszem", …) before translating correctly. Full limitations in the repo README.

Training data

Europarl v7 parallel corpus (Koehn, 2005), downloaded by the training scripts — not redistributed here. Held-out validation: 400 blocks of 5 consecutive sentences per direction.

Author

Marvin Uhlmann — https://github.com/Marvel4U · Code: MIT.