nllb-sango-finetuned-600m

MEYNG/nllb-sango-finetuned-600m

Open translation model fine-tuned specifically for Sango — the co-official language of the Central African Republic (with French, 5M+ speakers).

Fine-tuned from facebook/nllb-200-distilled-600M on 500,000+ Sango-French sentence pairs.

What is Sango?

Sango (ISO 639-1: sg, ISO 639-3: sag) is the co-official language of the Central African Republic (with French), spoken as a primary lingua franca by over 5 million people. It is a creole derived from the Ngbandi language, using a Latin orthography with tonal diacritics (â, ê, ë, î, ö, ô, û, ü).

Google added Sango to Google Translate in May 2026. This model is an open, Sango-specific fine-tune that you can run, inspect and adapt yourself — released openly so the community can build on it.

Model description

Benchmarks

Note on BLEU for Sango: BLEU penalises fluent paraphrases. chrF is more informative for Sango's morphological structure. Both are reported for completeness.

BENCH-001 — In-distribution (NLLB val.jsonl, N=200)

Evaluated on a held-out sample of the quality-filtered NLLB Sango-French corpus (N=200 sentences, LASER score ≥ 1.0, target language ID confidence ≥ 0.9). Same distribution as training data — measures specialised quality on the task the model was trained for.

French → Sango

Sango → French

Full results: scripts/training/results/bench001_threeway.json

---

BENCH-002 — Out-of-domain (FLORES-200 devtest, N=200) — Added 2026-05-22

Evaluated on FLORES-200 devtest (Wikipedia-style sentences, bri25yu Parquet mirror). Out-of-distribution relative to NLLB training corpus — measures generalisation to formal/encyclopaedic text.

French → Sango

Sango → French

Finding: The model overfit to the NLLB corpus style. On Wikipedia-style formal text it regresses below the baseline, including hallucination on long complex sentences. Use this model for text stylistically similar to its training domain (news / conversational parallel text). For general-purpose Sango translation, the vanilla NLLB-200 baseline is safer on out-of-domain inputs.

This finding drives the next priority: a diverse multi-domain Sango corpus for v2 training.

Full results: scripts/training/results/bench002_flores200.json

Usage

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
import torch

model_id = "MEYNG/nllb-sango-finetuned-600m"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSeq2SeqLM.from_pretrained(model_id, torch_dtype=torch.float16)

def translate(text, src_lang, tgt_lang, max_length=128):
    tokenizer.src_lang = src_lang
    inputs = tokenizer(text, return_tensors="pt", padding=True, truncation=True, max_length=max_length)
    forced_bos_token_id = tokenizer.convert_tokens_to_ids(tgt_lang)
    with torch.no_grad():
        generated = model.generate(
            **inputs,
            forced_bos_token_id=forced_bos_token_id,
            max_length=max_length,
            num_beams=4,
        )
    return tokenizer.batch_decode(generated, skip_special_tokens=True)[0]

# French → Sango
print(translate("Bonjour, comment allez-vous?", "fra_Latn", "sag_Latn"))

# Sango → French
print(translate("Singila mingi, balao.", "sag_Latn", "fra_Latn"))

Training data

Training data was sourced from the NLLB-200 parallel corpus (publicly available at storage.googleapis.com/allennlp-data-bucket/nllb/) and quality-filtered with two thresholds:

• LASER score ≥ 1.0 (semantic alignment quality)
• Target language identification confidence ≥ 0.9 (ensures the target is actually Sango)

Kept pairs: ~500,000+ (out of ~600,000 raw pairs in the NLLB Sango-French file).

Training was bidirectional: each sentence pair was included twice (FR→SAG and SAG→FR), giving ~1.1M training examples.

Training configuration

Seq2SeqTrainingArguments(
    num_train_epochs=3,
    per_device_train_batch_size=12,
    gradient_accumulation_steps=11,    # effective batch = 132
    learning_rate=5e-5,
    warmup_steps=500,
    bf16=True,                         # A100 native bfloat16
    predict_with_generate=False,       # bidirectional val set
    metric_for_best_model="eval_loss",
    hub_strategy="checkpoint",         # push to HF every 500 steps
)

Full training script: scripts/training/train_on_azure.py

Known limitations

• Domain specialisation (primary limitation): The model was trained exclusively on NLLB corpus text (web-crawled news and conversational parallel sentences). BENCH-002 (FLORES-200 devtest, 2026-05-22) shows it regresses below the vanilla baseline on Wikipedia-style formal text (FR→SAG: −2.72 BLEU). It also exhibits hallucination and repetition on long, complex out-of-domain sentences. Best suited for: news, conversational, and general informal French-Sango translation. Not recommended for: formal, legal, medical, or encyclopaedic text.
• Coverage: The NLLB Sango-French corpus is the only source of training data. Translation quality reflects the distribution of topics in that corpus.
• Sango orthography: Sango has some inconsistency in diacritic use across documents. The model inherits this from the training data.
• Low-resource baseline: Sango is a very low-resource language. Even after fine-tuning, expect lower absolute BLEU than high-resource pairs. chrF is a more reliable quality indicator for Sango.
• SAG→EN: This model was not trained on Sango-English pairs. For Sango→English, use a two-step pipeline: sag_Latn → fra_Latn → eng_Latn.
• Evaluation: BENCH-001 uses a held-out split of the training corpus (in-distribution). BENCH-002 uses FLORES-200 devtest (out-of-domain). See the Benchmarks section for both.

Related resources

Citation

@misc{wenezoui2026nllb-sango,
  author = {WENEZOUI, Michel},
  title = {MEYNG/nllb-sango-finetuned-600m: an open translation model fine-tuned specifically for Sango},
  year = {2026},
  publisher = {HuggingFace},
  url = {https://huggingface.co/MEYNG/nllb-sango-finetuned-600m}
}

License

CC BY-NC 4.0 (non-commercial). This license is inherited from the facebook/nllb-200-distilled-600M base model, which is released under CC BY-NC 4.0. Free for research, academic, NGO, government, and humanitarian use. Commercial use requires a separate arrangement with Meta (for the base model) and MEYNG (for the fine-tuned weights).

About MEYNG

MEYNG builds AI infrastructure for African languages. This model is part of a systematic effort to build open NLP resources for Sango, including:

• An open, structured Sango vocabulary dataset (590 verified entries, CC BY-SA 4.0)
• A Sango NLP toolkit (tokenizer, stemmer, language detection) — MIT-licensed, not yet released
• A deployed translation and learning platform at sangoai.sbs
• eNdara, an SMS and WhatsApp learning platform for the Central African Republic — live at e-ndara.org, early-stage