jakma-darija-A-adapter

jakma-darija-A-adapter

LoRA adapter for Qwen2.5-1.5B-Instruct fine-tuned on Moroccan Darija conversational data, used by the jak.ma Moroccan home-services platform.

What it does

Generates short, helpful replies in Moroccan Darija (Arabic script) for everyday home-services questions — plumber prices in Tangier, finding a painter in Casablanca, washing-machine repair quotes, etc. Built as a commercial-safe (Track A) variant that excludes CC-BY-NC-ND data.

How to use

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained('Qwen/Qwen2.5-1.5B-Instruct')
base = AutoModelForCausalLM.from_pretrained(
    'Qwen/Qwen2.5-1.5B-Instruct',
    torch_dtype=torch.bfloat16,
    device_map='cuda',
)
model = PeftModel.from_pretrained(base, 'Samielakkad1/jakma-darija-A-adapter')

msg = [{'role':'user','content':'شحال كيسوا تركيب صنبور؟'}]
prompt = tok.apply_chat_template(msg, tokenize=False, add_generation_prompt=True)
out = model.generate(**tok(prompt, return_tensors='pt').to('cuda'), max_new_tokens=200)
print(tok.decode(out[0], skip_special_tokens=True))

Training details

Training data

Validation: 6,665. Test: 6,665. Quality filter dropped 537. Deduplication dropped 755. Upsampling weights bias toward hand-crafted in-domain examples.

Intended use

• Powering the jak.ma Moroccan home-services platform conversational layer.
• Research baseline for Moroccan Darija LLM work at the 1–2 B parameter scale.

Limitations

• Small base model (1.5 B). Not competitive with frontier models on open-ended reasoning.
• Adapter is 1.94 GB because the trainer saved embed_tokens + lm_head alongside the LoRA weights. Load with peft.PeftModel.from_pretrained — the loader handles vocab alignment.
• Trained on a curated Track A corpus; may underperform on topics outside home-services.

Author

Sami El Akkad (@Samielakkad1) — MSc, Tsinghua SIGS — building jak.ma.

Source

Training code: https://github.com/Samielakkad/jakma