nllb-mongolian

🇲🇳 NLLB-600M Mongolian v2.0 (Major Release)

A state-of-the-art bidirectional translation model for Mongolian (Cyrillic) ↔ English, fine-tuned from Meta's NLLB-200 (600M Distilled).

Version 2.0 is a complete overhaul trained on over 711,800 parallel sentence pairs with dedicated native idiomatic and technical anchor injection. It completely eliminates the Russian loanword bias, broken literal calques, and hallucinated grammar common in older or baseline models.

---

🚀 Key Improvements in v2.0

---

💻 Quick Start (Python / Transformers)

import torch
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

model_id = "Tsengel-mn/nllb-mongolian"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSeq2SeqLM.from_pretrained(model_id).to("cuda" if torch.cuda.is_available() else "cpu")

def translate(text: str, direction: str = "en-mn") -> str:
    if direction == "en-mn":
        src, tgt = "eng_Latn", "khk_Cyrl"
    else:
        src, tgt = "khk_Cyrl", "eng_Latn"
        
    tokenizer.src_lang = src
    inputs = tokenizer(text, return_tensors="pt").to(model.device)
    
    tgt_id = tokenizer.convert_tokens_to_ids(tgt)
    with torch.no_grad():
        tokens = model.generate(
            **inputs,
            forced_bos_token_id=tgt_id,
            max_length=128,
            num_beams=2
        )
    return tokenizer.decode(tokens[0], skip_special_tokens=True)

# English -> Mongolian
print(translate("The database migration completed successfully.", "en-mn"))
# Output: Мэдээллийн сангийн нүүлгэн шилжилт амжилттай дууссан.

# Mongolian -> English
print(translate("Өглөөний мэнд, та сайн байна уу?", "mn-en"))
# Output: Good morning, how are you today?

---

🛠️ Training Specifications

• Base Architecture: Meta NLLB-200 Distilled (600M parameters, Seq2Seq Encoder-Decoder).
• Training Corpus: 711,821 pairs:
  • 703,221 verified clean parallel sentences.
  • Curated idiomatic, conversational, and technical Mongolian anchor sets oversampled 200x.
• Hardware: NVIDIA GeForce RTX 4060 (8GB VRAM) on AMD Ryzen 9 7900.
• Fine-Tuning Method: PEFT LoRA (Rank 64, Alpha 128 across all attention projections and MLP feed-forwards: q_proj, k_proj, v_proj, out_proj, fc1, fc2).
• Optimization: AdamW 8-bit (adamw_bnb_8bit), Cosine Learning Rate schedule with 300 warmup steps, BF16 mixed precision.
• Training Duration: 8,000 steps (~2.25 hours).
• Final Validation Loss: 0.9372.
• Export Format: Standalone fused FP16/BF16 weights (model.safetensors).

---

🌐 Language Codes (NLLB Flores-200)

When calling the tokenizer, use standard NLLB language identifiers:

• Mongolian (Cyrillic): khk_Cyrl
• English: eng_Latn

---

📜 License & Citation

This model is licensed under Meta's Creative Commons Attribution-NonCommercial 4.0 International (CC-BY-NC 4.0) license in alignment with the underlying NLLB-200 weights.

Maintained by Tsengel-mn for the Mongolian open-source AI language ecosystem.