Downloads · 30 days
0
XSkills/nllb-200-turkmen-english-lora-adapter
nllb-200-turkmen-english-lora-adapter is a translation model from XSkills. Use it when you need text moved from one language to another. It is set up for transformers. The card lists the license as cc-by-nc-4.0.
Author : Merdan Durdyyev Base model : facebook/nllb-200-distilled-600M Tuning method : Low-Rank Adaptation (LoRA) on only the qproj & vproj matrices (≈ 2.4 M trainable → 0.38 % of total params).
Downloads · 30 days
0
Access
Public
Updated Apr 28, 2025
Repo size
123 MB
Likes
1
Public
Click a slice to open those files.
.json258 MB · 53%
From the Hugging Face model README
Author : Merdan Durdyyev
Base model : facebook/nllb-200-distilled-600M
Tuning method : Low-Rank Adaptation (LoRA) on only the q_proj & v_proj matrices (≈ 2.4 M trainable → 0.38 % of total params).
I built this checkpoint as the final project for my Deep-Learning class and as a small contribution to the Turkmen AI community, where open-source resources are scarce.
Try it on Space demo Article with full technical journey is available Medium.
| Direction | BLEU ↑ | chrF ↑ | TER ↓ | Test pairs |
|---|---|---|---|---|
| tk → en | 26.07 | 52.97 | 68.39 | 50 |
| en → tk | 8.13 | 39.39 | 87.30 | 50 |
| Metric | Fine-tuned | Original |
|---|---|---|
| BLEU | 26.07 | 26.48 |
| chrF | 52.97 | 52.91 |
| TER | 68.39 | 69.70 |
| Metric | Fine-tuned | Original |
|---|---|---|
| BLEU | 26.07 | 26.48 |
| chrF | 52.97 | 52.91 |
| TER | 68.39 | 69.70 |
Scores computed with sacre BLEU 2.5, chrF, TER on the official test split.
A separate spreadsheet with human adequacy/fluency ratings is available in the article.
If you want to use merged model visit nllb-200-turkmen-english-lora
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
from peft import PeftModel
BASE = "facebook/nllb-200-distilled-600M"
ADAPTER = "XSkills/nllb-200-turkmen-english-lora-adapter"
tok = AutoTokenizer.from_pretrained(BASE)
base = AutoModelForSeq2SeqLM.from_pretrained(BASE)
model = PeftModel.from_pretrained(base, ADAPTER) # ← attaches the LoRA weights
def tr(text, src="tuk_Latn", tgt="eng_Latn"):
tok.src_lang = src
ids = tok(text, return_tensors="pt", truncation=True, max_length=128)
out = model.generate(
**ids,
forced_bos_token_id=tok.convert_tokens_to_ids(tgt),
max_length=128,
num_beams=5
)
return tok.decode(out[0], skip_special_tokens=True)
print(tr("Men kitaby okaýaryn.")) # → “I am reading the book.”
| Item | Value |
|---|---|
| GPU | 1 × NVIDIA A100 40 GB (Google Colab) |
| Wall-time | ~ 3 minutes |
| Optimiser | AdamW |
| Learning rate | 1 × 10⁻⁵, cosine schedule, warm-up 10% |
| Epochs | 5 |
| Batch size | 4 (train) / 8 (eval) |
| Weight-decay | 0.005 |
| FP16 | Yes |
| LoRA config | r=16, alpha=32, dropout=0.05, modules = ["q_proj","v_proj"] |
lora_config = LoraConfig(
r=16,
lora_alpha=32,
target_modules=["q_proj", "v_proj"],
lora_dropout=0.05,
bias="none",
task_type=TaskType.SEQ_2_SEQ_LM,
)
training_args = Seq2SeqTrainingArguments(
output_dir=FINETUNED_DIR,
per_device_train_batch_size=4,
per_device_eval_batch_size=8,
weight_decay=0.005,
save_total_limit=3,
learning_rate=1e-5,
num_train_epochs=5,
lr_scheduler_type="cosine",
predict_with_generate=True,
fp16=True if torch.cuda.is_available() else False,
logging_dir="./logs",
logging_steps=50,
eval_steps=50,
save_steps=100,
eval_accumulation_steps=2,
report_to="tensorboard",
warmup_ratio=0.1,
metric_for_best_model="eval_bleu", # Use BLEU for model selection
greater_is_better=True,
)
Automatic metrics are given in TL;DR. A manual review on 50 random test sentences showed:
@misc{durdyyev2025turkmenNLLBLoRA,
title = {LoRA Fine‐tuning of NLLB‐200 for Turkmen–English Translation},
author = {Durdyyev, Merdan},
year = {2025},
url = {https://huggingface.co/XSkills/nllb-200-turkmen-english-lora-adapter}
}
If you have questions, suggestions or want to collaborate, please reach out through e-mail, LinkedIn or Telegram.