nllb-200-kapampangan-en

Got it 👍 Thanks for sharing your Wav2Vec2 model card format — I’ll now draft your NLLB fine-tuned model card in the same style and structure for consistency.

---

Model Card for NLLB-200-Distilled-600M-Kapampangan-English

This model is a fine-tuned version of facebook/nllb-200-distilled-600M for machine translation between Kapampangan and English. It was trained on a custom dataset of 1,685 sentence pairs and achieves competitive results on a held-out test split. The model supports bidirectional translation between Kapampangan (pam) and English (en).

---

Model Details

Model Description

• Developed by: Sean Almendral
• Model type: Machine Translation (NMT)
• Languages (NLP): Kapampangan (kap), English (en)
• License: Apache-2.0
• Finetuned from model: facebook/nllb-200-distilled-600M
• Pipeline tag: translation

Model Sources

• Repository: Hugging Face Model Card
• Paper [base model]: No Language Left Behind: Scaling Human-Centered Machine Translation

---

Uses

Direct Use

• Translating between Kapampangan ↔ English.
• Educational and cultural preservation of Kapampangan.
• Baseline translation model for further fine-tuning.

Downstream Use

• Integration into ASR + MT pipelines (e.g., Kapampangan speech → English text).
• Language learning tools, chatbots, or educational apps.

Out-of-Scope Use

• Not suited for medical, legal, or safety-critical translations.
• May produce inaccurate outputs for low-resource domains or idiomatic expressions.

---

Bias, Risks, and Limitations

• Dataset size is relatively small (1,685 pairs), so coverage is limited.
• May not generalize well to informal speech, slang, or rare words.
• Risk of bias toward dataset domains (conversational & general sentences).
• Limited exposure to code-switching between Kapampangan and English.

Recommendations

• Validate translations before sensitive use.
• Extend training with larger and domain-specific corpora for production systems.

---

How to Get Started with the Model

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("kruokruo/nllb-200-kapampangan-en")
model = AutoModelForSeq2SeqLM.from_pretrained("kruokruo/nllb-200-kapampangan-en")

text = "Ali ku balu"
inputs = tokenizer(text, return_tensors="pt")

translated_tokens = model.generate(
    **inputs,
    forced_bos_token_id=tokenizer.lang_code_to_id["eng_Latn"]
)
translation = tokenizer.decode(translated_tokens[0], skip_special_tokens=True)

print(translation)  # "I didn’t know"

---

Training Details

Training Data

• Custom dataset of 1,685 parallel Kapampangan–English pairs.
• Preprocessing included lowercasing and punctuation normalization.

Training Procedure

• Base model: facebook/nllb-200-distilled-600M
• Optimizer: AdamW
• Learning rate: 1e-5
• Epochs: 5 (with Early Stopping)
• Train/Validation split: 80/20 (seed=42)

Training Hyperparameters

• Precision: fp32
• Batch size: GPU-dependent (with gradient accumulation)
• Scheduler: Linear decay with warmup

---

Evaluation

Testing Data

• 20% held-out split from the dataset.

Metrics

• Training Loss: 1.3471 → 0.0322
• Validation Loss: ↓ 1.3589 → 1.2495 (epoch 2), ↑ 1.5447 (epoch 5)
• SacreBLEU: ↑ 16.6973 → 29.6054

Sample Prediction

• Source: malyari meng gisyan ing kahun ba yang ibusni.
• Prediction: The box was about to be removed.
• Reference: you can tear the box open.

BLEU Breakdown (test):

• BLEU = 0.4046
• Precisions = [61.79%, 42.34%, 35.20%, 31.20%]
• BP = 0.9826
• Translation length = 2,387
• Reference length = 2,429

SacreBLEU Breakdown (test):

• SacreBLEU = 40.46
• Counts = [1475, 868, 603, 430]
• Totals = [2387, 2050, 1713, 1378]
• Precisions = [61.79%, 42.34%, 35.20%, 31.20%]
• BP = 0.9826

---

Environmental Impact

• Hardware Type: NVIDIA L4 GPU (Google Colab Pro)
• Training Duration: [fill in if tracked]
• Energy Consumed: ~104.87 gCO2e (estimated using ML Impact calculator)
• Energy Needed: ~257.03 Wh
• Cloud Provider: Google Colab Pro

---

Technical Specifications

Model Architecture and Objective

• Architecture: NLLB-200-Distilled-600M (600M parameters)
• Objective: Fine-tuned for supervised sequence-to-sequence translation

Compute Infrastructure

• Framework: PyTorch + Hugging Face Transformers
• Trainer: Hugging Face Seq2SeqTrainer API

---

Citation

@misc{nllb-kapampangan-en,
  title={NLLB-200-Distilled-600M-Kapampangan-English: Machine Translation Model},
  author={Sean Almendral},
  year={2025},
  howpublished={\url{https://huggingface.co/kruokruo/nllb-200-kapampangan-en}},
}

---

Model Card Authors

Sean Almendral

Model Card Contact

📧 [email protected]

---