nllb-rus-tyv-v3-dict_16.5k

nllb-rus-tyv-v3-dict_16.5k

Russian ↔ Tyvan (Tuvan) translation model. Fine-tuned from facebook/nllb-200-distilled-600M with an aggressively pruned vocabulary: from ~256k to 16.5k subword pieces (keeping only Russian/Tyvan tokens seen in the corpus, plus Latin letters, digits, punctuation, and special tokens). This cuts the model from 615M to 370M parameters (−40%) while improving quality over the full-vocabulary baseline.

Tyvan language code: tyv_Cyrl (added; embedding initialized from Kyrgyz kir_Cyrl). Russian: rus_Cyrl.

Results (test, chrF++ / BLEU, greedy)

Evaluated on a held-out test set of 2,000 pairs (seed=42), identical for both systems.

The pruned model is smaller and slightly more accurate on all four metrics. chrF++ is the primary metric (character-level, more reliable than BLEU for agglutinative Turkic languages).

Usage

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

model_id = "Agisight/nllb-rus-tyv-v3-dict_16.5k"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSeq2SeqLM.from_pretrained(model_id)

def translate(text, src, tgt):
    tok.src_lang = src
    inputs = tok(text, return_tensors="pt")
    out = model.generate(**inputs, forced_bos_token_id=tok.convert_tokens_to_ids(tgt),
                         max_new_tokens=128, num_beams=1)
    return tok.batch_decode(out, skip_special_tokens=True)[0]

print(translate("Привет, как дела?", "rus_Cyrl", "tyv_Cyrl"))
print(translate("Экии!", "tyv_Cyrl", "rus_Cyrl"))

Training

• Base: facebook/nllb-200-distilled-600M
• Data: Agisight/tyv-rus-200k (~296k pairs)
• Vocabulary pruning: SentencePiece pieces restricted to those appearing in the corpus plus ASCII letters/digits/punctuation; embeddings and final logits transferred to the pruned vocab; tyv_Cyrl added and initialized from kir_Cyrl.
• Objective: standard seq2seq cross-entropy, both directions, Adafactor (lr 1e-4), batch 48, max_len 96, greedy eval, early-stopping on dev chrF++ plateau.
• Best checkpoint: dev avg chrF++ 48.45 (step 40,000).

Intended use & limitations

• Designed for lightweight / on-device ru↔tyv translation (web, CPU, Apple Silicon).
• Out-of-vocabulary characters (other scripts, emoji) fall back to byte tokens or <unk> — acceptable for the ru/tyv/Latin scenario it targets.
• Trained on short sentences; very long inputs are truncated at 96 tokens during training.
• Not a general multilingual model — only Russian ↔ Tyvan.

Citation

If you use this model, please cite the dataset and base model (NLLB-200, Meta AI).