tumbuka_nllb-200

---

language:

• tum
• eng tags:
• translation
• seq2seq
• nllb
• nllb-200
• instruction-tuning
• malawi license: mit datasets:
• mwanzau/tumbuka-language-malawi metrics:
• bleu model-index:
• name: tumbuka_nllb-200 results: []

---

tumbuka_nllb-200 (Version 2.0)

This model is a highly optimized, fine-tuned checkpoint of Meta's facebook/nllb-200-distilled-600M, specifically adapted for high-fidelity translation and instruction-following tasks in the Tumbuka (Chitumbuka) language, spoken primarily in Northern Malawi and Eastern Zambia.

While Version 1.0 laid the groundwork for basic translation, Version 2.0 fixes deep-seated regional language leakage (such as Nyanja/Chichewa loan-words) by undergoing 9+ hours of advanced multi-stage training. It combines premium English-Tumbuka translation pairs with dedicated Tumbuka instruction-tuning datasets.

Key Improvements in V2.0:

• Linguistic Purity: Eradicated common baseline model bleed (e.g., automatically corrects baseline defaults like Takulandilani into pure Tumbuka Mwapokeleleka).
• Contextual Precision: Significantly improves technical and situational vocabulary mapping (e.g., shifting the translation of "reliable irrigation" from rigid variations to natural maji ghakugomezgeka ghakuthira).

---

Model Description

• Developed by: SaintsStudios
• Model type: Encoder-Decoder (Seq2Seq)
• Language(s) (NLP): English (eng_Latn) to Tumbuka (tum_Latn), and Tumbuka Monolingual Instruction Tasking
• License: MIT
• Finetuned from model: facebook/nllb-200-distilled-600M

---

Performance Examples

---

How to Use the Model

You can use this model directly with the Hugging Face transformers library for inference.

Python Inference Code

import torch
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

model_id = "SaintsStudios/tumbuka_nllb-200"

print("Loading model and tokenizer...")
tokenizer = AutoTokenizer.from_pretrained("facebook/nllb-200-distilled-600M")
model = AutoModelForSeq2SeqLM.from_pretrained(model_id)

device = "cuda" if torch.cuda.is_available() else "cpu"
model.to(device)

def translate_to_tumbuka(text: str):
    # Set the language configs
    tokenizer.src_lang = "eng_Latn"
    inputs = tokenizer(text, return_tensors="pt").to(device)
    
    # Force target generation to start with the Tumbuka language token
    forced_bos_token_id = tokenizer.convert_tokens_to_ids("tum_Latn")
    
    translated_tokens = model.generate(
        **inputs,
        forced_bos_token_id=forced_bos_token_id,
        max_length=128,
        num_beams=4,
        early_stopping=True
    )
    
    return tokenizer.decode(translated_tokens[0], skip_special_tokens=True)

# Test run
sample_text = "Let us learn together."
output = translate_to_tumbuka(sample_text)
print(f"English: {sample_text}")
print(f"Tumbuka: {output}") 
# Output: Tiyeni tisambire pamoza


Training Hyperparameters (V2.0)
Dataset Blend: 50% Translation Pairs / 50% Monolingual Alpaca Instruction Datasets

- Epochs: 3
- Learning Rate: 2e-5
- Optimizer: Adafactor
- Batch Size: 2 (Gradient Accumulation Steps: 8)
- Precision: FP16 Mixed Precision
- Hardware: Kaggle Cloud Compute (GPU T4)

Acknowledgements
Special thanks to the open-source contributors maintaining datasets for Malawian localized languages, making high-quality regional adaptation pipelines possible.