Downloads · 30 days
0
Touqeer19/kashmiri-english-nllb-lora
kashmiri-english-nllb-lora is a machine learning model from Touqeer19. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Downloads · 30 days
0
Access
Public
Updated Aug 13, 2026
Repo size
83.4 MB
Likes
0
Public
Click a slice to open those files.
.json32.2 MB · 77%
From the Hugging Face model README
License apache-2.0
Language en,ks
Base_model facebook/nllb-200-distilled-600M
Tags translation,kashmiri,nllb,lora,peft
Datasets ai4bharat/BPCC
Metrics bleu,chrf
Kashmiri-English Translator (NLLB-200 + LoRA) A LoRA fine-tuned version of facebook/nllb-200-distilled-600M for English-Kashmiri (Perso-Arabic script) translation.
Also available as a merged, standalone model (no peft required to load): Touqeer19/kashmiri-english-nllb-merged
Model description This model fine-tunes NLLB-200-distilled-600M using LoRA (Low-Rank Adaptation) on 5,573 English-Kashmiri sentence pairs. It translates between English (eng_Latn) and Kashmiri written in Perso-Arabic script (kas_Arab).
Base model: facebook/nllb-200-distilled-600M Fine-tuning method: LoRA (r=16, alpha=32, targeting q_proj/v_proj) Training data: 5,573 pairs (90% split) from AI4Bharat's BPCC nllb-seed configuration, Kashmiri Arabic-script subset Trainable parameters: 2,359,296 (0.17% of total 1.4B model parameters) Intended use Experimental English-Kashmiri machine translation. Given the small fine-tuning dataset, this is best understood as a research/portfolio demonstration of a low-resource-language fine-tuning pipeline, not a production-grade translator.
Evaluation Evaluated on a held-out test set of 310 sentence pairs, unseen during training.
Model BLEU chrF Baseline (nllb-200-distilled-600M, no fine-tuning) 11.57 44.68 Fine-tuned (this model) 11.94 45.32 Improvement +0.37 +0.63 Honest note on these results: the improvement from fine-tuning is modest. This is expected — NLLB-200 already includes Kashmiri in its pretraining data, so this fine-tune is refining an already-competent model on a small (5,573-example) dataset, rather than teaching the model a new language from scratch. The fine-tuned model reliably produces fluent, coherent Kashmiri output, confirmed via round-trip translation checks, but the quantitative gain over the strong baseline is small.
How to use python from transformers import AutoModelForSeq2SeqLM, AutoTokenizer from peft import PeftModel
base_model_name = "facebook/nllb-200-distilled-600M" adapter_name = "Touqeer19/kashmiri-english-nllb-lora"
tokenizer = AutoTokenizer.from_pretrained(adapter_name) model = AutoModelForSeq2SeqLM.from_pretrained(base_model_name) model = PeftModel.from_pretrained(model, adapter_name) model.eval()
def translate(text, src_lang="eng_Latn", tgt_lang="kas_Arab"): tokenizer.src_lang = src_lang inputs = tokenizer(text, return_tensors="pt") generated_tokens = model.generate( **inputs, forced_bos_token_id=tokenizer.convert_tokens_to_ids(tgt_lang), max_length=128 ) return tokenizer.decode(generated_tokens[0], skip_special_tokens=True)
print(translate("Hello, how are you?"))
Important: forced_bos_token_id must be set explicitly to the target language token, or generation will default to an unrelated language.
Prefer not to deal with peft? Use the merged version instead — same weights, loads with plain transformers.
Try it live A public demo is available at Touqeer19/kashmiri-english-translator.
Training procedure 3 epochs, learning rate 1e-4, effective batch size 16 (batch size 4 x gradient accumulation 4) Padding tokens masked with -100 in labels so they don't contribute to loss Mixed precision (fp16) Trained on a single T4 GPU (Google Colab, free tier) Limitations Small training set (5,573 pairs) drawn from a single domain (Wikipedia-style encyclopedic text via NLLB-Seed) — performance on conversational or domain-specific text is untested. Only Perso-Arabic script (kas_Arab) was used for fine-tuning; Devanagari Kashmiri (kas_Deva) was not included. Sensitive to input casing. ALL CAPS input produces noticeably degraded output compared to standard sentence casing, since training data was entirely standard-cased. Evaluated with automatic metrics (BLEU/chrF) only; no native-speaker human evaluation has been conducted yet. License Apache 2.0 (matching the base NLLB-200 model's license). Training data from AI4Bharat's BPCC is used under its documented terms (CC0 / CC BY 4.0 depending on subset).