english-german-transformer

PyM Studio English-German Transformer

A 63.1M-parameter encoder-decoder Transformer created from scratch by Kamron M (Hugging Face: kmehrad, Kaggle: kammehr) at PyM Studio.

The model translates English text to German. It was trained on the WMT14 English-German recipe and uses a joint 37,000-token byte-level BPE tokenizer. The published weights average five validation-best checkpoints. The best constituent validation loss was 2.9379.

Architecture

• 6 encoder and 6 decoder layers
• 512-dimensional embeddings
• 8 attention heads
• 256-token source and target context
• sinusoidal positional encodings
• shared input/output token weights
• approximately 63.1M unique trainable parameters

Files

• averaged.pt: averaged PyTorch model weights and configuration
• modeling_transformer.py: inference model definition
• translate_interactive.py: beam-search translation CLI
• tokenizer.json: joint English-German tokenizer
• requirements.txt: minimal runtime dependencies
• LICENSE: Apache License 2.0

Usage

pip install -r requirements.txt
python translate_interactive.py

Or translate one sentence:

python translate_interactive.py "The cat is sleeping"

Use --device cpu or --device cuda to choose a device. Beam search defaults to beam size 4 and length penalty 0.6.

Attribution and license

Copyright 2026 Kamron M (Kaggle: kammehr), PyM Studio.

Licensed under the Apache License, Version 2.0. See LICENSE.