overlay-translator-nllb-int8

NLLB-200-distilled-600M, game-chat abbreviations (CTranslate2 int8)

An int8-quantized CTranslate2 build of facebook/nllb-200-distilled-600M fine-tuned to handle English game-chat abbreviations and jargon. It is the local, offline translation backend of Overlay-Translator, a Windows app that translates text you select on screen.

Always pin a revision (tag); main follows the newest upload.

What changed, English → Russian (lines not seen in training):

What it was trained to do

English game-chat terms come in two kinds:

• translate (177 abbreviations): the meaning is carried over: brb, idk, ty, tbh, lmk, omw, afaik, nvm, lfm, wtt, oom, rez ...
• keep (1,101 terms): gaming terms that players write the same way in every language, so the translation must keep them verbatim: gg, wp, glhf, afk, xd, pvp, dps, aoe, hp, cd, cc, ult, lfg, wts, wtb, op, nerf, gank, adc, rng, ilvl, bis, number suffixes like 500g, 2kk ...

In addition, 768 ordinary English words from the glossary (armor, sword, kill) were trained as plain negatives that must be translated normally, so the model does not learn to leave every short word untranslated.

Only English → de, fr, es, ru, uk, be, pl, cs, hu was trained. Other directions are unchanged from the base model.

Training

Data (23,575 training pairs):

As in v1, each pair maps the phrase with the abbreviation to a translation of the expanded phrase (lfm +10, need tank and dps → translation of looking for more +10, need tank and dps, keeping +10 and dps). The targets were translated by the base NLLB model itself (fp32, beam 4); keep terms that NLLB lost were restored by masking them as [n] and translating again. A phrase that consists only of keep terms and punctuation (sks!) is its own target.

Method: LoRA (r=32, α=64, dropout 0.05) on the q/k/v/out projections of every self- and cross-attention block (9.4 M trainable parameters, 1.5 %), trained from the base model (not on top of v1-abbr), using 🤗 transformers 5.17 + peft 0.21. 5 epochs, effective batch 16, lr 2e-4 with linear decay, bf16, group-by-length, seed 42, about 2 hours on one RTX 3060 Ti. The checkpoint with the lowest validation loss (step 6,600 of 7,370) was merged into the base weights (merge_and_unload) and converted the same way as v0-base:

ct2-transformers-converter --model nllb-finetuned-v2-merged \
    --output_dir nllb-200-ct2-int8 --quantization int8 \
    --copy_files tokenizer.json tokenizer_config.json special_tokens_map.json \
                 sentencepiece.bpe.model generation_config.json

All files except model.bin are byte-for-byte identical to v0-base and v1-abbr, so the revisions are drop-in replacements for each other.

Evaluation

All models were run as int8 CTranslate2 on the CPU, beam 4. The metric is chrF (sacrebleu 2.6, β=2); the confidence intervals come from a paired bootstrap (95 %, 1000 resamples).

v1 test set (1,301 pairs, unchanged from v1-abbr): one unseen phrase per v1 abbreviation (911 pairs) and held-out FLORES-200 dev sentences (390 pairs).

Glossary test set (1,539 pairs: 171 held-out phrases × 9 languages: template sentences, held-out glossary examples and plain words).

Per target language, v1 test set:

The general per-language scores rest on about 43 sentences each and are noisy; the pooled number is reliable.

How to read the scores. The references are NLLB translations of the expanded phrases (and of the isolated plain words), not human translations. The abbreviation chrF shows whether the abbreviation was understood, not how good the translation is. The plain-word references penalise valid alternatives (kill → «убить» vs «убивать») and a few are broken; without the single worst reference the plain-word chrF is 86.0 for v0-base, 54.4 for v1-abbr and 66.6 for v2-game.

Usage

pip install ctranslate2 sentencepiece huggingface_hub

import ctranslate2
import sentencepiece as spm
from huggingface_hub import snapshot_download

model_dir = snapshot_download("Alhite/overlay-translator-nllb-int8", revision="v2-game")

translator = ctranslate2.Translator(model_dir, device="cpu", compute_type="int8")
# GPU: ctranslate2.Translator(model_dir, device="cuda", compute_type="int8_float16")
sp = spm.SentencePieceProcessor(model_file=f"{model_dir}/sentencepiece.bpe.model")

def translate(text: str, src: str, tgt: str, beam_size: int = 4) -> str:
    # NLLB input format: <src_lang> tokens... </s>; the target language is forced as a prefix.
    source = [src] + sp.encode(text, out_type=str) + ["</s>"]
    result = translator.translate_batch([source], target_prefix=[[tgt]], beam_size=beam_size)
    tokens = result[0].hypotheses[0][1:]  # drop the target-language token
    return sp.decode(tokens)

print(translate("omw to the boss, wait for me", "eng_Latn", "rus_Cyrl"))

NLLB uses FLORES-200 language codes: eng_Latn, deu_Latn, fra_Latn, spa_Latn, rus_Cyrl, ukr_Cyrl, bel_Cyrl, pol_Latn, ces_Latn, hun_Latn.

Size and speed are the same as v0-base: the architecture and the quantization did not change (model.bin 594 MiB; about 0.4 s per short chat phrase on 4 CPU threads, beam 4).

Limitations

• Ordinary words are sometimes left untranslated. On isolated plain words v2-game scores below v0-base (96 of 513 left in English vs 53 for v0-base); it recovers a large part of what v1-abbr lost, but not all of it.
• Unseen variants are expanded wrongly, e.g. lf2m for mythic dungeon → «Я хочу вернуться в мифический подземелье» (only lf1m is in the training data).
• Keep terms can still be lost in some contexts: rng finally blessed you → «Господь благословил тебя»; overall about 4–6 % of keep terms are lost.
• The model may expand unknown short tokens (names, other slang) as if they were abbreviations.
• The training targets were produced by NLLB itself, so its mistakes were learned too: tank → «резервуар», clutch → «сцепление», healer → «врач».
• Only English as the source language was trained.
• Inherited from NLLB-200: a research model, not intended for production deployment, domain-specific (medical, legal) or document translation, or certified translations.

License

Model weights: CC-BY-NC-4.0. Non-commercial use only.

This model is an adaptation of facebook/nllb-200-distilled-600M by Meta AI (NLLB Team), licensed CC-BY-NC-4.0, and is distributed under the same license. Changes made to the original weights:

1. fine-tuned with LoRA on the data described above, then merged into the weights;
2. converted to the CTranslate2 format and quantized to int8.

The tokenizer files (sentencepiece.bpe.model, tokenizer*.json, special_tokens_map.json) and generation_config.json are copied unchanged from the original model. This model is not affiliated with or endorsed by Meta. It is provided "as is", without warranties of any kind (see section 5 of the license).

Training data

None of the training data is redistributed in this repository; it was only used for training and evaluation.

Citation

@article{nllb2022,
  title   = {No Language Left Behind: Scaling Human-Centered Machine Translation},
  author  = {{NLLB Team} and Costa-jussà, Marta R. and Cross, James and others},
  journal = {arXiv preprint arXiv:2207.04672},
  year    = {2022}
}

@article{goyal2022flores,
  title   = {The {FLORES}-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation},
  author  = {Goyal, Naman and Gao, Cynthia and Chaudhary, Vishrav and Chen, Peng-Jen and
             Wenzek, Guillaume and Ju, Da and Krishnan, Sanjana and Ranzato, Marc'Aurelio and
             Guzm{\'a}n, Francisco and Fan, Angela},
  journal = {Transactions of the Association for Computational Linguistics},
  volume  = {10},
  pages   = {522--538},
  year    = {2022}
}

@inproceedings{hu2022lora,
  title     = {{LoRA}: Low-Rank Adaptation of Large Language Models},
  author    = {Hu, Edward J. and Shen, Yelong and Wallis, Phillip and Allen-Zhu, Zeyuan and
               Li, Yuanzhi and Wang, Shean and Wang, Lu and Chen, Weizhu},
  booktitle = {International Conference on Learning Representations},
  year      = {2022}
}

@software{speer2022wordfreq,
  author    = {Speer, Robyn},
  title     = {rspeer/wordfreq: v3.0},
  publisher = {Zenodo},
  year      = {2022},
  doi       = {10.5281/zenodo.7199437}
}