Downloads · 30 days
13
23% of all-time downloads
Sorjin1-LoRA
muzaffercky/Sorjin1-LoRA
Sorjin1-LoRA is a text generation model from muzaffercky. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as apache-2.0.
The raw LoRA adapter weights from continual pretraining of Qwen/Qwen2.5-7B on Kurdish Kurmanji academic theses.
All-time downloads
56
Public
Repo size
657 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors646 MB · 98%
At a glance
- Task
- Text Generation
- Library
- peft
- License
- apache-2.0
- Access
- Public
- Created
- Jun 6, 2026
- Updated
- Jul 17, 2026
- SHA
- 85f95ebc
Base models
Model card
From the Hugging Face model README
Sorjîn1-LoRA
The raw LoRA adapter weights from continual pretraining of Qwen/Qwen2.5-7B on Kurdish Kurmanji academic theses.
Use this if you want to apply the Kurdish language adapter to a different Qwen2.5-7B variant, or retrain from this checkpoint.
Usage
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-7B", torch_dtype="bfloat16")
model = PeftModel.from_pretrained(base, "muzaffercky/Sorjin1-LoRA")
# Merge adapter into base weights
model = model.merge_and_unload()
Training Details
- Base model: Qwen/Qwen2.5-7B
- Dataset: muzaffercky/kurdish-kurmanji-theses
- LoRA rank: 64 | Alpha: 128 | Dropout: 0.05
- Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
- Trainable parameters: 161,480,704 (2.08% of total)
- Final train loss: 2.057 | Final eval loss: 1.8552
Model Family
| Model | Description |
|---|---|
| Sorjîn1-LoRA (this model) | Raw LoRA adapter weights |
| Sorjîn1-7B | Continually pretrained base model |
| Sorjîn1-7B-Instruct | Merged with Qwen2.5-7B-Instruct |
- Task
- Text Generation
- Library
- peft
- License
- apache-2.0
- Languages
- ku, kmr
- Created
- Jun 6, 2026
- Updated
- Jul 17, 2026