Downloads · 30 days
6
13% of all-time downloads
GoldenNet-Qwen2.5-0.5B-LoRA-v1
Alamori/GoldenNet-Qwen2.5-0.5B-LoRA-v1
GoldenNet-Qwen2.5-0.5B-LoRA-v1 is a text generation model from Alamori. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
<div align="center" <img src="https://img.shields.io/badge/Golden%20Net%20AI-Iraqi%20Gov%20NLP-gold?style=for-the-badge" alt="Golden Net AI"/ <img src="https://img.shields.io/badge/Method-LoRA-purple?style=for-the-bad…
All-time downloads
45
Public
Parameters
494M
1000 MB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors988 MB · 98%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- qwen2
- Access
- Public
- Created
- Jan 28, 2026
- Updated
- Jan 28, 2026
- SHA
- 69b5525b
Base models
Datasets
Model card
From the Hugging Face model README
GoldenNet-Qwen2.5-0.5B-LoRA-v1
<div align="center"> <img src="https://img.shields.io/badge/Golden%20Net%20AI-Iraqi%20Gov%20NLP-gold?style=for-the-badge" alt="Golden Net AI"/> <img src="https://img.shields.io/badge/Method-LoRA-purple?style=for-the-badge" alt="LoRA"/> <img src="https://img.shields.io/badge/Language-Arabic-green?style=for-the-badge" alt="Arabic"/> </div>Model Description
GoldenNet-Qwen2.5-0.5B-LoRA-v1 is a LoRA fine-tuned version of Qwen/Qwen2.5-0.5B-Instruct specialized for Iraqi Government Correspondence Processing.
This is the standard LoRA variant (no quantization), offering potentially better quality than QLoRA at the cost of higher VRAM usage during training.
Tasks
- Document Classification - 8 categories (طلب، شكوى، تقرير، إعلام، استفسار، دعوة، تعميم، إحالة)
- Named Entity Recognition - Extracts persons, organizations, locations, dates, monetary values, laws
Model Comparison
| Model | Method | Train Loss | Eval Loss | Training Time |
|---|---|---|---|---|
| QLoRA-v1 | 4-bit QLoRA | 0.448 | 0.2998 | 49s |
| LoRA-v1 | Standard LoRA | 0.496 | 0.3665 | 70s |
Training Details
| Parameter | Value |
|---|---|
| Base Model | Qwen/Qwen2.5-0.5B-Instruct |
| Fine-tuning Method | LoRA (no quantization) |
| LoRA Rank | 64 |
| LoRA Alpha | 128 |
| LoRA Dropout | 0.05 |
| Learning Rate | 1e-4 |
| Epochs | 3 |
| Batch Size | 1 (effective: 16) |
| Max Sequence Length | 2048 |
| Precision | BF16 |
| Trainable Parameters | 35.2M (6.6%) |
Loss Progression
- Epoch 1: 0.979
- Epoch 2: 0.350
- Epoch 3: 0.247
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained(
"Alamori/GoldenNet-Qwen2.5-0.5B-LoRA-v1",
device_map="auto",
torch_dtype="auto"
)
tokenizer = AutoTokenizer.from_pretrained("Alamori/GoldenNet-Qwen2.5-0.5B-LoRA-v1")
# Classification example
correspondence = """جمهورية العراق
وزارة التربية
العدد: 1234/ت/2025
إلى/ السيد مدير عام التعليم المحترم
م/ طلب تعيين معلمين
نرجو الموافقة على تعيين 50 معلماً.
مع التقدير"""
instruction = "صنّف المراسلة الحكومية التالية إلى إحدى الفئات: طلب، شكوى، تقرير، إعلام، استفسار، دعوة، تعميم، إحالة. أجب بصيغة JSON."
messages = [{"role": "user", "content": f"{instruction}\n\n{correspondence}"}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.1)
print(tokenizer.decode(outputs[0][inputs['input_ids'].shape[1]:], skip_special_tokens=True))
Related Models
- GoldenNet-Qwen2.5-0.5B-QLoRA-v1 - 4-bit quantized version
License
Apache 2.0
<div align="center"> <b>Developed by Golden Net AI</b><br> <i>Empowering Iraqi Government Digital Transformation</i> </div>
- Task
- Text Generation
- Library
- transformers
- Type
- qwen2
- License
- apache-2.0
- Languages
- ar, en
- Created
- Jan 28, 2026
- Updated
- Jan 28, 2026