chokri-nllb-finetuned

Chokri–English Translation Model

Fine-tuned version of Meta's NLLB-200 distilled 600M for Chokri (nri) ↔ English translation — the first machine translation model for the Chokri language.

Language

Chokri (ISO 639-3: nri, Glottocode: chok1243) is a Tibeto-Burman language spoken by approximately 100,000 people in Phek District, Nagaland, northeast India. It belongs to the Angami sub-group of the Kuki-Chin-Naga branch branch of the Sino-Tibetan language family and was not supported by any prior machine translation system.

Model Details

How to Use

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

model_name = "knbliss/chokri-nllb-finetuned"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSeq2SeqLM.from_pretrained(model_name)

# Chokri → English
# Note: model uses "lus_Latn" (Mizo) as a proxy token for Chokri during training
inputs = tokenizer("U nü bazo ho.", return_tensors="pt", src_lang="lus_Latn")
translated = model.generate(
    **inputs,
    forced_bos_token_id=tokenizer.convert_tokens_to_ids("eng_Latn"),
    num_beams=4,
    max_new_tokens=128,
    no_repeat_ngram_size=3,
    repetition_penalty=1.5
)
print(tokenizer.batch_decode(translated, skip_special_tokens=True))

# English → Chokri
inputs = tokenizer("Where are you going?", return_tensors="pt", src_lang="eng_Latn")
translated = model.generate(
    **inputs,
    forced_bos_token_id=tokenizer.convert_tokens_to_ids("lus_Latn"),
    num_beams=4,
    max_new_tokens=128,
    no_repeat_ngram_size=3,
    repetition_penalty=1.5
)
print(tokenizer.batch_decode(translated, skip_special_tokens=True))

Important: Because nri (Chokri) was absent from NLLB-200's original vocabulary, this model uses lus_Latn (Mizo) as a proxy language token. Mizo is a closely related Tibeto-Burman language that also uses Latin script, making it the best available proxy in the NLLB token space.

Training

Fine-tuned from facebook/nllb-200-distilled-600M on ~8,000 verified human-translated Chokri–English sentence pairs. The primary training corpus is the Chokri New Testament (Bible Society of India), supplemented by community-contributed sentence pairs collected through a live data pipeline with active contributors and reviewers.

Training data is tagged by village and region to capture dialect variation across Phek District, Nagaland.

Live Demo

Try the model at the Chokri English Translator — a live web interface where you can translate text and contribute new sentence pairs to improve the dataset.

Community & Ongoing Work

This model is part of an active community-led data collection effort:

• Contributors submit new Chokri–English sentence pairs via the translator interface or a community form
• Reviewers verify submissions for accuracy and naturalness before they enter the training dataset
• Institutional partner: Chakhesang Chokri Literature Board (CCLB), the official language authority for Chokri
• Dataset target: Growing toward 40,000+ sentence pairs to reach the "low-resource" tier in multilingual MT benchmarks

The project is listed in the OLDI Awesome New Languages in Machine Translation initiative, and Chokri is an active contribution target in BOUQuET, Meta's multilingual evaluation benchmark.

Limitations

• Training data is predominantly from a single domain (Bible corpus), which may reduce performance on everyday or technical text
• The Bible translation used (NAG5BSI) reflects one regional dialect and has not been formally adopted as the CCLB standard
• Chokri has significant dialect variation across villages;
• Tone is not marked in the practical orthography used for training data, which may cause occasional ambiguity

Acknowledgements

Built on Meta's NLLB-200 open model. Institutional support from the Chakhesang Chokri Literature Board (CCLB). Thanks to all community contributors and reviewers who continue to grow the dataset.

Citation

If you use this model in your research, please cite:

@misc{nienu2025chokri,
  author    = {Nienu, Kuyi},
  title     = {Chokri--English Translation Model},
  year      = {2025},
  publisher = {Hugging Face},
  url       = {https://huggingface.co/knbliss/chokri-nllb-finetuned}
}