zh-en-transformer

Chinese-English Transformer

A Chinese-to-English machine translation model trained from scratch with a standard four-layer Pre-LN Transformer and separate SentencePiece BPE tokenizers.

Model

• Encoder layers: 4
• Decoder layers: 4
• Hidden size: 256
• Attention heads: 8
• FFN size: 1024
• Maximum sequence length: 64
• Source vocabulary: 16,000
• Target vocabulary: 16,000
• Parameters: 15,565,824
• Decoding: beam search (beam 4, length penalty 1.2) with per-layer KV caching

Training Data

The model was trained on 94,644 filtered Chinese-English sentence pairs from the OPUS-100 training split. The official OPUS-100 validation and test splits were kept separate. Users should review the licenses of the underlying OPUS corpora before redistribution or commercial use.

Evaluation

Evaluation uses greedy decoding with SacreBLEU on the complete filtered official splits.

On an NVIDIA GeForce RTX 5060 Laptop GPU, cached decoding measured 21.8 ms per sentence at batch size 1 and 111.0 ms per batch at batch size 16 (median, maximum output length 64). This is approximately 1.22x and 1.70x faster than the uncached reference implementation, respectively.

Beam 4 / LP 1.2 measured 26.6 ms P50 and 34.4 ms P95 at batch size 1. At batch size 16 it measured 189.4 ms P50 and 223.6 ms P95.

Usage

pip install -r requirements.txt
python evaluate_mt.py --text "今天天气很好。"

HTTP Service

Install the serving dependencies and start one model worker:

pip install -r requirements.txt
python serve.py --host 0.0.0.0 --port 8000

POST /translate accepts either one string or a list. The response always contains a list in the same order as the input:

curl -X POST http://127.0.0.1:8000/translate \
  -H "Content-Type: application/json" \
  -d '{"texts":["你好。","今天天气很好。"]}'

Concurrent requests are dynamically coalesced for up to 8 ms, with a default maximum batch size of 16. Tune MT_DEVICE, MT_MAX_BATCH_SIZE, MT_BATCH_WAIT_MS, MT_MAX_QUEUE_SIZE, MT_REQUEST_TIMEOUT_S, MT_MAX_TEXT_CHARS, and MT_MAX_REQUEST_ITEMS through environment variables. Use MT_DECODE=greedy for the lower-latency mode; the default accurate mode uses MT_DECODE=beam, MT_NUM_BEAMS=4, and MT_LENGTH_PENALTY=1.2. Set MT_API_TOKEN to require Authorization: Bearer <token> on translation requests. GET /health reports queue depth and observed batch statistics.

The service returns 413 for oversized input, 429 when the inference queue is full, and 504 when a request exceeds the inference timeout.

The implementation and checkpoint use a custom PyTorch model rather than the Transformers AutoModel API.

Limitations

• The training corpus is relatively small and contains some noisy or misaligned sentence pairs.
• The model is intended as an educational/research baseline, not a production translation service.
• Long inputs are truncated to 64 SentencePiece tokens.
• Names, numbers, colloquial dialogue, and rare domains may be translated inaccurately.