eo-mt-v13-tiny-bidir

eo-mt-v13-tiny-bidir

MarianMT bidirectional English<->Esperanto, ~69M params (mt/configs/tiny.yaml).

Fresh-training baseline on the new v13 corpus, which drops the OPUS-100 KDE/GNOME/Ubuntu contamination that caused the v12 UI-label collapse failure mode (@ info: whatsthis -> & Resize, Name -> A tool to store a new image).

Config (tiny.yaml)

d_model 512, 6 enc + 6 dec layers, 8 heads, ffn 2048. Total: 69.2M / trainable 68.7M params.

Data

• jensjepsen/esperanto-mt-parallel-v13 (5.03M rows; no opus-100)
• jensjepsen/esperanto-mt-math-parallel-v2 (sentences + rows)
• jensjepsen/esperanto-mt-yago-parallel-v2 (labels + comments)
• jensjepsen/esperanto-orca-math-gemini-parallel

Total pool: 5.84M rows/epoch.

Training

Single RTX 5090, bf16, 3 epochs (136,962 steps), batch_size=128 (no grad accum), peak LR 5e-4, cosine decay, warmup 2000 steps, weight_decay 0.01, label smoothing 0.1, dropout 0.1, direction=bidir, val-direction=en2eo. Runtime 4h 32m.

Tokenizer: spm_eneo_48k_v3.model (SPM unigram, case-preserving nmt_nfkc).

Final val BLEU (cased sacrebleu, ep 3.0)

vs eo-mt-v12-bidir (large 285M, opus-100 contaminated): 95-100% of v12's BLEU across all sets, with 4.5x fewer params, and cleanly free of the KDE-junk regression on capitalized-fragment inputs.

Loading

from transformers import MarianMTModel
import sentencepiece as spm

model = MarianMTModel.from_pretrained("jensjepsen/eo-mt-v13-tiny-bidir")
sp = spm.SentencePieceProcessor(model_file="spm_eneo_48k_v3.model")
# Prepend <eo> (id=48001) to input for EN->EO, or <en> (id=48000) for EO->EN.