GoldenNet-Qwen2.5-3B-QLoRA-v1

GoldenNet-Qwen2.5-3B-QLoRA-v1

<div align="center"> <img src="https://img.shields.io/badge/Golden%20Net%20AI-Iraqi%20Gov%20NLP-gold?style=for-the-badge" alt="Golden Net AI"/> <img src="https://img.shields.io/badge/Method-QLoRA-blue?style=for-the-badge" alt="QLoRA"/> <img src="https://img.shields.io/badge/Model-3B-orange?style=for-the-badge" alt="3B"/> <img src="https://img.shields.io/badge/Language-Arabic-green?style=for-the-badge" alt="Arabic"/> </div>

Model Description

GoldenNet-Qwen2.5-3B-QLoRA-v1 is a QLoRA fine-tuned version of Qwen/Qwen2.5-3B-Instruct specialized for Iraqi Government Correspondence Processing.

This is the larger 3B model variant, offering improved performance over the 0.5B models while remaining efficient for edge deployment.

Tasks

1. Document Classification - 8 categories (طلب، شكوى، تقرير، إعلام، استفسار، دعوة، تعميم، إحالة)
2. Named Entity Recognition - Extracts persons, organizations, locations, dates, monetary values, laws

Model Comparison

Training Details

Loss Progression

• Epoch 0.5: 1.143
• Epoch 1.0: 0.462
• Epoch 1.5: 0.295
• Epoch 2.0: 0.244
• Epoch 2.5: 0.192
• Epoch 3.0: 0.181 (final)

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "Alamori/GoldenNet-Qwen2.5-3B-QLoRA-v1",
    device_map="auto",
    torch_dtype="auto"
)
tokenizer = AutoTokenizer.from_pretrained("Alamori/GoldenNet-Qwen2.5-3B-QLoRA-v1")

# Classification example
correspondence = """جمهورية العراق
وزارة التربية
العدد: 1234/ت/2025

إلى/ السيد مدير عام التعليم المحترم

م/ طلب تعيين معلمين

نرجو الموافقة على تعيين 50 معلماً.

مع التقدير"""

instruction = "صنّف المراسلة الحكومية التالية إلى إحدى الفئات: طلب، شكوى، تقرير، إعلام، استفسار، دعوة، تعميم، إحالة. أجب بصيغة JSON."

messages = [{"role": "user", "content": f"{instruction}\n\n{correspondence}"}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.1)
print(tokenizer.decode(outputs[0][inputs['input_ids'].shape[1]:], skip_special_tokens=True))

When to Use This Model

• Use 3B-QLoRA-v1 for best accuracy when you have sufficient VRAM (~6GB for inference)
• Use 0.5B-QLoRA-v1 for fast inference on constrained hardware
• The 3B model shows ~16% improvement in eval loss over the 0.5B models

Related Models

• GoldenNet-Qwen2.5-0.5B-QLoRA-v1 - Smaller, faster variant
• GoldenNet-Qwen2.5-0.5B-LoRA-v1 - Standard LoRA
• GoldenNet-Qwen2.5-0.5B-Full-v1 - Full fine-tune

License

Apache 2.0

---

<div align="center"> <b>Developed by Golden Net AI</b><br> <i>Empowering Iraqi Government Digital Transformation</i> </div>