nllb_khasi_sft

NLLB-200-1.3B — per-pair SFT for English → Khasi

NLLB-200-1.3B fine-tuned on English → Khasi alone, rather than jointly with the other three language pairs.

This is the counterpart to randomDude26/nllb_mktg_sft, which is trained jointly on Mizo, Khasi, Tulu and Gondi. For Khasi, the joint adapter collapses and this one does not:

A 3.7× BLEU gain from removing the other three languages. Khasi is Austroasiatic and has no relative among NLLB's supported languages, so it appears to be crowded out during joint training — while Mizo, Tulu and Gondi all benefit from it. This is the main reason the model exists.

A LoRA adapter — load it on top of facebook/nllb-200-1.3B.

Results

Decontaminated eng→khasi test set (n = 800). BLEU / chrF++ via sacrebleu (BLEU(effective_order=True), CHRF(word_order=2)).

* base NLLB has no kha_Latn token; the zero-shot number forces eng_Latn as a proxy and is not a native translation.

Checkpoint used for these scores: nllb-sft-en-khasi-20260521_100238/final_adapter

Note that a decoder-only model (Gemma) still beats this by a wide margin on Khasi. Per-pair training rescues NLLB from collapse but does not close the gap.

Other directions

This adapter is English→Khasi only. It was trained on nothing else and degrades badly if pushed at other pairs:

Use nllb_mktg_sft for Mizo, Tulu or Gondi.

Training data

English–Khasi split of randomDude26/mi_kh_tulu_dataset:

Validation and test are decontaminated: no exact or near duplicate of any test/val sentence appears in training (exact pair, exact source, exact target, ≥0.90 character similarity, ≥0.80 4-gram containment, ≥0.80 token Jaccard).

Configuration

LoRA

A kha_Latn language token was added to the tokenizer — base NLLB-200 has no Khasi token. The trained embedding for it ships inside the adapter's lm_head (NLLB ties input embeddings and the LM head), so the adapter is self-contained.

Training

Usage

import torch
from transformers import AutoTokenizer
from transformers.models.m2m_100 import M2M100ForConditionalGeneration
from peft import PeftModel

REPO = "randomDude26/nllb_khasi_sft"

# the tokenizer in this repo carries the added kha_Latn token
tokenizer = AutoTokenizer.from_pretrained(REPO)

base = M2M100ForConditionalGeneration.from_pretrained(
    "facebook/nllb-200-1.3B", dtype=torch.bfloat16
)
base.resize_token_embeddings(len(tokenizer), mean_resizing=False)

model = PeftModel.from_pretrained(base, REPO).merge_and_unload().cuda().eval()

tokenizer.src_lang = "eng_Latn"
inputs = tokenizer("The temple is very old.", return_tensors="pt").to("cuda")

out = model.generate(
    **inputs,
    forced_bos_token_id=tokenizer.convert_tokens_to_ids("kha_Latn"),
    num_beams=4, max_new_tokens=256,
)
print(tokenizer.batch_decode(out, skip_special_tokens=True)[0])

Citation

Paper under anonymous review. Dataset: randomDude26/mi_kh_tulu_dataset.