Downloads · 30 days
18
0% of all-time downloads
XSkills/nllb-200-turkmen-english-lora
nllb-200-turkmen-english-lora is a translation model from XSkills. Use it when you need text moved from one language to another. It is set up for transformers. The card lists the license as cc-by-nc-4.0.
Author : Merdan Durdyyev Base model : facebook/nllb-200-distilled-600M Tuning method : Low-Rank Adaptation (LoRA) on only the qproj & vproj matrices (≈ 2.4 M trainable → 0.38 % of total params).
Downloads · 30 days
18
0% of all-time downloads
All-time downloads
12.7K
Public
Parameters
615M
2.5 GB on disk
Likes
2
Public
Click a slice to open those files.
.safetensors2.5 GB · 98%
From the Hugging Face model README
Author : Merdan Durdyyev
Base model : facebook/nllb-200-distilled-600M
Tuning method : Low-Rank Adaptation (LoRA) on only the q_proj & v_proj matrices (≈ 2.4 M trainable → 0.38 % of total params).
I built this checkpoint as the final project for my Deep-Learning class and as a small contribution to the Turkmen AI community, where open-source resources are scarce.
Try it on Space demo Article with full technical journey is available Medium.
| Metric | Fine-tuned | Original | Difference |
|---|---|---|---|
| BLEU | 8.24 | 8.12 | +0.12 |
| chrF | 39.55 | 39.46 | +0.09 |
| TER (lower is better) | 87.20 | 87.30 | -0.10 |
| Metric | Fine-tuned | Original | Difference |
|---|---|---|---|
| BLEU | 25.88 | 26.48 | -0.60 |
| chrF | 52.71 | 52.91 | -0.20 |
| TER (lower is better) | 67.70 | 69.70 | -2.00 |
Scores computed with sacre BLEU 2.5, chrF, TER on the official test split.
A separate spreadsheet with human adequacy/fluency ratings is available in the article.
(If you want to take a look to the LoRA adapter visit nllb-200-turkmen-english-lora-adapter)
Using piplene
from transformers import pipeline
# Create the translation pipeline
pipe = pipeline("translation", model="XSkills/nllb-200-turkmen-english-lora")
# Translate from English to Turkmen
# You need to specify the source and target languages using their FLORES-200 codes
text = "Hello, how are you today?"
translated = pipe(text, src_lang="eng_Latn", tgt_lang="tuk_Latn")
print(translated)
Using Tokenizer
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
model_id = "XSkills/nllb-200-turkmen-english-lora"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSeq2SeqLM.from_pretrained(model_id)
def tr(text, src="tuk_Latn", tgt="eng_Latn"):
tok.src_lang = src
ids = tok(text, return_tensors="pt", truncation=True, max_length=128)
out = model.generate(
**ids,
forced_bos_token_id=tok.convert_tokens_to_ids(tgt),
max_length=128,
num_beams=5
)
return tok.decode(out[0], skip_special_tokens=True)
print(tr("Men kitaby okaýaryn."))
| Item | Value |
|---|---|
| GPU | 1 × NVIDIA A100 40 GB (Google Colab) |
| Wall-time | ~ 3 minutes |
| Optimiser | AdamW |
| Learning rate | 1 × 10⁻⁵, cosine schedule, warm-up 10% |
| Epochs | 5 |
| Batch size | 4 (train) / 8 (eval) |
| Weight-decay | 0.005 |
| FP16 | Yes |
| LoRA config | r=16, alpha=32, dropout=0.05, modules = ["q_proj","v_proj"] |
lora_config = LoraConfig(
r=16,
lora_alpha=32,
target_modules=["q_proj", "v_proj"],
lora_dropout=0.05,
bias="none",
task_type=TaskType.SEQ_2_SEQ_LM,
)
training_args = Seq2SeqTrainingArguments(
output_dir=FINETUNED_DIR,
per_device_train_batch_size=4,
per_device_eval_batch_size=8,
weight_decay=0.005,
save_total_limit=3,
learning_rate=1e-5,
num_train_epochs=5,
lr_scheduler_type="cosine",
predict_with_generate=True,
fp16=True if torch.cuda.is_available() else False,
logging_dir="./logs",
logging_steps=50,
eval_steps=50,
save_steps=100,
eval_accumulation_steps=2,
report_to="tensorboard",
warmup_ratio=0.1,
metric_for_best_model="eval_bleu", # Use BLEU for model selection
greater_is_better=True,
)
Automatic metrics are given in TL;DR. A manual review on 50 random test sentences showed:
We welcome contributions to improve Turkmen-English translation capabilities! Here's how you can help:
All contributors will be acknowledged in the model documentation. Contact [email protected] with any questions or to discuss potential contributions.
Note: This model is licensed under CC-BY-NC-4.0, so all contributions must be compatible with non-commercial use only.
@misc{durdyyev2025turkmenNLLBLoRA,
title = {LoRA Fine‐tuning of NLLB‐200 for Turkmen–English Translation},
author = {Durdyyev, Merdan},
year = {2025},
url = {https://huggingface.co/XSkills/nllb-200-turkmen-english-lora}
}
If you have questions, suggestions or want to collaborate, please reach out through e-mail, LinkedIn or Telegram.