Downloads · 30 days
0
GzimJusufi/albanian-qwen2.5-7b-instruct
albanian-qwen2.5-7b-instruct is a machine learning model from GzimJusufi. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
The first instruction-tuned large language model for the Albanian language — spoken by ~7-8 million people across Albania, Kosovo, North Macedonia, and the diaspora, yet severely underserved in NLP.
Downloads · 30 days
0
Access
Public
Updated Jul 3, 2026
Repo size
496 MB
Likes
0
Public
Click a slice to open those files.
.safetensors162 MB · 93%
From the Hugging Face model README
The first instruction-tuned large language model for the Albanian language — spoken by ~7-8 million people across Albania, Kosovo, North Macedonia, and the diaspora, yet severely underserved in NLP.
This model applies the same fine-tuning methodology used for the Macedonian LLM to Albanian — a second underserved Balkan language. Fine-tuned using QLoRA on Albanian text, then instruction-tuned for conversational use.
| Property | Details |
|---|---|
| Base model | Qwen/Qwen2.5-7B-Instruct |
| Fine-tuning method | QLoRA (4-bit) via Unsloth (2x faster training) |
| Training data | 10,000 Albanian documents (Wikipedia) |
| Instruction dataset | 230 Albanian instruction pairs |
| Topics covered | Geography, history, culture, food, language, translation, reasoning |
| Training time | ~3 hours on Tesla T4 |
| Developer | Gzim Jusufi — CS student, North Macedonia |
| Model | Perplexity |
|---|---|
| Qwen2.5-7B (base) | 18.74 |
| GzimJusufi/albanian-qwen2.5-7b | 7.88 |
| Improvement | 58% |
The higher base perplexity (18.74 vs 10.64 for Macedonian) reflects that Albanian was even more underserved in the base model's pretraining data — making this improvement more meaningful.
| Language | Speakers | Base PPL | Fine-tuned PPL | Improvement |
|---|---|---|---|---|
| Macedonian | ~2M | 10.64 | 5.21 | 51% |
| Albanian | ~7-8M | 18.74 | 7.88 | 58% |
Same methodology, stronger result on the more underserved language.
History: User: Kush ishte Skënderbeu? Assistant: Skënderbeu (1405-1468) ishte një prijës ushtarak shqiptar i shekullit 15, i njohur për mbrojtjen e territoreve shqiptare kundër Perandorisë Osmane.
Independence: User: Çfarë bëri Ismail Qemali më 28 nëntor 1912? Assistant: Më 28 nëntor 1912, Ismail Qemali shpalli pavarësinë e Shqipërisë në Vlorë, duke u bërë kryetar i qeverisë së parë të vendit.
Culture: User: Çfarë është byreku? Assistant: Byreku është një byrekë tradicionale e bërë me shtresa brumi të hollë të mbushura me djathë, mish ose spinaq, e pjekur në furrë.
Translation: User: Përkthe 'mirupafshim' në anglisht. Assistant: Mirupafshim në anglisht përkthehet 'goodbye'.
Reasoning: User: Sa vjet jetoi Skënderbeu? Assistant: Skënderbeu lindi në 1405 dhe vdiq në 1468, duke jetuar rreth 63 vjet.
from unsloth import FastModel
model, tokenizer = FastModel.from_pretrained(
model_name="GzimJusufi/albanian-qwen2.5-7b-instruct",
max_seq_length=2048,
)
messages = [
{"role": "user", "content": "Kush ishte Skënderbeu?"}
]
inputs = tokenizer.apply_chat_template(
messages, tokenize=True,
add_generation_prompt=True,
return_tensors="pt"
).to("cuda")
outputs = model.generate(input_ids=inputs, max_new_tokens=80)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Trained on the albanian-instructions dataset — 230 carefully curated Albanian instruction-response pairs covering:
@misc{jusufi2025albanian, author = {Jusufi, Gzim}, title = {Albanian Qwen2.5-7B Instruct: First Instruction-Tuned LLM for Albanian}, year = {2026}, publisher = {HuggingFace}, url = {https://huggingface.co/GzimJusufi/albanian-qwen2.5-7b-instruct} }
Built with ❤️ for the Albanian language and NLP community