qwen2.5-1.5b-luganda-lora

Qwen2.5-1.5B + QLoRA — English → Luganda

LoRA adapter for Qwen/Qwen2.5-1.5B-Instruct fine-tuned for instruction-following translation into Luganda.

Load the adapter

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")
model = PeftModel.from_pretrained(base, "SsemuliJoseph/qwen2.5-1.5b-luganda-lora")
tok   = AutoTokenizer.from_pretrained("SsemuliJoseph/qwen2.5-1.5b-luganda-lora")

Training details

QLoRA fine-tuned adapter on Qwen2.5-1.5B-Instruct for English → Luganda translation, trained in chat/instruction format. Part of an industrial-training NLP pipeline at Sunbird AI, targeting low-resource Ugandan language translation.

This is a PEFT adapter, not a standalone model — load it on top of the base Qwen2.5-1.5B-Instruct weights.

Model Description

• Base model: Qwen/Qwen2.5-1.5B-Instruct
• Adapter type: LoRA (via QLoRA — 4-bit quantized base + LoRA fine-tuning)
• Task: Instruction-formatted English → Luganda translation
• Language(s): English (en), Luganda (lg)
• Developed by: SsemuliJoseph

Training Data

Fine-tuned on a chat-formatted version of the same Argilla-annotated golden dataset used for the companion baseline model (SsemuliJoseph/nllb-luganda-baseline), reviewed under an Accept / Needs Editing / Reject annotation discipline.

Evaluation

Compared qualitatively against the NLLB baseline (BLEU 17.84) on the same held-out golden set. See companion baseline model card for the quantitative baseline score; this adapter was evaluated side-by-side on identical rows rather than re-scored independently.

Intended Use

Research and prototyping for English–Luganda machine translation in an instruction-following / chat format, in low-resource settings. Demonstrated live in the luganda-translator-demo Space.

Limitations

• Trained on a small dataset; not production-grade.
• Not evaluated for bias, toxicity, or robustness across Luganda dialects.
• Should not be used for legal, medical, or safety-critical translation.
• Requires the base Qwen2.5-1.5B-Instruct model plus PEFT to load.

How to Use

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")
model = PeftModel.from_pretrained(base, "SsemuliJoseph/qwen2.5-1.5b-luganda-lora")

messages = [{"role": "user", "content": "Translate to Luganda: Hello, how are you?"}]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True)
outputs = model.generate(inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))