rnn-translator-tr-en-63M

Nematus RNN · Turkish → English (BPE)

An attentional RNN (bi-GRU encoder / GRU decoder) neural machine translation model that translates Turkish → English, trained with the Nematus toolkit on a ~142k-sentence news-domain parallel corpus using a joint 32k BPE subword vocabulary.

This is the RNN baseline for the machine translation models collection. Siblings: the stronger Transformer atahanuz/transformers-translator-tr-en-75M and the reverse-direction RNN atahanuz/rnn-translator-en-tr-63M.

TL;DR

• BLEU 35.10 on a 1,000-sentence held-out test set (beam 12, length-normalized).
• ~63M parameters · single A100-80GB · ~110 min training.
• A baseline: the Transformer sibling scores 42.78 on the same test set (see matrix below).

Model details

Training data

• Corpus: mt_datasets_vol2 — 144,065 Turkish–English sentence pairs (news / current affairs).
• Filtering: drop any pair where either side is empty or longer than 60 whitespace tokens → 143,926 pairs.
• Split (shuffled, seed 42): 141,926 train / 1,000 dev / 1,000 test.

Preprocessing

1. Tokenization: Moses tokenizer.perl -l <lang> -no-escape.
2. Joint BPE: 32,000 merges learned on the training side only (TR+EN concatenated), applied without a vocabulary-frequency threshold (it over-fragments on a corpus this size). Resulting subword vocab: TR 23,211 / EN 17,659 types.

(The BPE model is identical to the Transformer siblings — BPE is architecture-independent.)

Training configuration

• Optimizer: Adam (β1 0.9, β2 0.999, ε 1e-8), learning rate 1e-4 constant, gradient-norm clip 1.0.
• Batch 320 sentences (larger batches NaN'd for the RNN at this LR), maxlen 120, label smoothing 0.0.
• Validation every 200 updates; early stopping with patience 10 on dev cross-entropy.
• Hardware: 1× NVIDIA A100-80GB.

Training run

• Best validation at update 22,200 (dev cross-entropy 52.88) — the checkpoint in this repo.
• Early-stopped at update ~24,400. Wall-clock ≈ 110 min — notably longer than the Transformer (~68 min): the RNN needs more updates to converge and is slower per word (sequential recurrence vs the Transformer's parallel attention).

Evaluation

BLEU via multi-bleu.perl on the merged-BPE (Moses-tokenized) hypothesis vs the tokenized reference; beam 12, length-normalized.

Architecture × direction matrix (same 1,000 mirrored test pairs):

This model = RNN TR→EN = 35.10 (n-gram 65.5 / 41.3 / 28.2 / 19.9, BP 1.00). The Transformer beats the RNN by ~7–8 BLEU in both directions. The RNN produces fluent translations (unlike the project's earlier word-level RNN that mode-collapsed on a tiny corpus) but degrades on long / complex sentences.

Example

Files in this repo

• model.npz.* — Nematus/TensorFlow checkpoint (best-validation, update 22,200).
• train.bpe.tr.json, train.bpe.en.json — Nematus source/target vocabularies (referenced by model.npz.json).
• tr-en.bpe.codes, vocab.tr, vocab.en — the joint BPE model, used to segment new input.
• nematus_tf220_compat.patch — makes Nematus run on TensorFlow ≥ 2.16 / NumPy ≥ 2 / Python 3.12.

Installation

git clone https://github.com/EdinburghNLP/nematus.git
cd nematus && git checkout 49d050863bc9644b8c0a9d9ab6e54ccd30f927dd
git apply /path/to/nematus_tf220_compat.patch
pip install "tensorflow>=2.16" "numpy>=2" subword-nmt sacremoses
cd ..

The patch fixes graph-mode tf.debugging.assert_shapes (incl. the RNN beam-search path), NumPy-2 dtype=object, a numpy-float JSON encoder, the softmax-projection layer args, and the Keras-3 Dropout training-flag handling.

Usage

# 0. download this model
python3 -c "from huggingface_hub import snapshot_download; \
snapshot_download('atahanuz/rnn-translator-tr-en-63M', local_dir='tr_en_rnn')"

# 1. preprocess a Turkish input file (one sentence per line)
perl nematus/data/tokenizer.perl -l tr -no-escape < input.tr > input.tok.tr
subword-nmt apply-bpe -c tr_en_rnn/tr-en.bpe.codes < input.tok.tr > input.bpe.tr

# 2. translate — run from the model dir so the dict paths in model.npz.json resolve
cd tr_en_rnn
python3 ../nematus/nematus/translate.py -m model.npz -i ../input.bpe.tr -o ../out.bpe.en -k 12 -n -b 50
cd ..

# 3. postprocess: undo BPE, then detokenize
sed -E 's/(@@ )|(@@ ?$)//g' out.bpe.en > out.tok.en
python3 - <<'EOF'
from sacremoses import MosesDetokenizer
d = MosesDetokenizer(lang='en')
print('\n'.join(d.detokenize(l.split()) for l in open('out.tok.en', encoding='utf-8')))
EOF

(The model architecture is read from model.npz.json, so the exact same commands work as for the Transformer models — only the checkpoint differs.)

Intended use & limitations

• A research baseline; for best quality use the Transformer sibling (+7.7 BLEU).
• Domain: news / current affairs. Quality drops out of domain and on long / syntactically complex sentences.
• Trained on sentences ≤ 60 tokens.
• Tokenized multi-bleu.perl BLEU — for citable numbers use sacreBLEU on detokenized output.
• No safety/bias auditing.

License

cc-by-4.0 placeholder — set to match your training-data terms.

Acknowledgements

Nematus; subword-nmt. GRU attentional NMT: Bahdanau et al. (2015) / Sennrich et al. (Nematus).