ru-en-transformer

Russian → English Transformer (from scratch)

A compact encoder–decoder Transformer trained from scratch (no pretrained weights) for Russian→English translation. Built as a learning project — the tokenizer, model, training loop, and beam-search decoding are all hand-written.

• Parameters: ~11.5M
• Architecture: 4 encoder + 4 decoder layers, d_model=256, 8 heads, d_ff=1024, sinusoidal positional encoding, tied input/output embeddings
• Tokenizer: byte-level BPE, vocab 16,000 (shared RU/EN), included as tokenizer.json
• Data: 200,000 opus-100 RU–EN pairs
• Training: 60 epochs max, early-stopped ~epoch 40 (patience 5), Adam + Noam LR schedule, label smoothing 0.1, batch size 64

Results (held-out test split, 1,951 sentences)

Validation BLEU was 26.96. Note that opus-100 (subtitle-derived) contains some misaligned reference pairs, so these BLEU numbers slightly underestimate true quality.

Usage

# pip install torch tokenizers huggingface_hub
from huggingface_hub import snapshot_download
import sys

path = snapshot_download("prplguyy/ru-en-transformer")
sys.path.insert(0, path)
from translator import translate

print(translate("Привет, как у тебя дела сегодня?", method="beam"))
# -> "Hey, how are you doing today?"

The repo bundles everything needed to run inference on CPU: model.pt (weights), tokenizer.json, and the model/decoding code (config.py, model.py, decoding.py, translator.py).

Limitations

Small from-scratch model: strong on everyday conversational sentences, but expect rough edges on rare proper names, idioms, and long or technical text. English→Russian is not supported (trained one direction only).

Links

• 🕹️ Live demo: https://transformertranslaterussian2english.streamlit.app/
• 💻 Source / training code: https://github.com/prplguyy/transformerTranslateRussianEnglish