Downloads · 30 days
4
6% of all-time downloads
arabic-llm-synthetic-500k
ManarAlrabie/arabic-llm-synthetic-500k
arabic-llm-synthetic-500k is a text generation model from ManarAlrabie. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as other.
LoRA adapter for ALLaM-7B-Instruct-preview, fine-tuned on the human-curated Arabic instruction dataset InstAr-500k under a fixed budget of 500K training tokens. One of six adapters from a controlled study comparing hu…
All-time downloads
67
Public
Repo size
160 MB
Likes
1
Trending 1
Repo makeup
Click a slice to open those files.
.safetensors160 MB · 97%
At a glance
- Task
- Text Generation
- Library
- peft
- License
- other
- Access
- Public
- Created
- Feb 3, 2026
- Updated
- Jun 18, 2026
- SHA
- 64bb0127
Base models
Datasets
Model card
From the Hugging Face model README
ALLaM-7B-Instruct — Curated 500K
LoRA adapter for ALLaM-7B-Instruct-preview, fine-tuned on the
human-curated Arabic instruction dataset InstAr-500k under a fixed
budget of 500K training tokens. One of six adapters from a
controlled study comparing human-curated versus synthetic Arabic
instruction data under matched token budgets.
Model Details
- Base model: humain-ai/ALLaM-7B-Instruct-preview
- Adapter type: LoRA (QLoRA, 4-bit NF4)
- Training data: InstAr-500k
- Token budget: 500K tokens
- Language: Arabic
Training Configuration
| Setting | Value |
|---|---|
| Quantization | 4-bit NF4 (QLoRA) |
| LoRA rank / alpha | 16 / 32 |
| LoRA dropout | 0.05 |
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Optimizer | Paged AdamW (8-bit) |
| Learning rate | 2e-4 |
| LR scheduler | cosine, 100 warmup steps |
| Epochs | 3 |
| Effective batch size | 16 (2 × 8 grad. accum.) |
| Max sequence length | 512 |
| Precision | fp16 |
| Seed | 42 |
| Hardware | NVIDIA A100 (40GB) |
Evaluation
Evaluated with lm-evaluation-harness on seven Arabic benchmarks (ACVA 5-shot; others zero-shot). Accuracy:
| Benchmark | Score |
|---|---|
| Arab Culture | 0.361 |
| AlGhafa | 0.594 |
| AraDiCE | 0.590 |
| ACVA | 0.770 |
| Arabic Exams | 0.508 |
| ArabicMMLU | 0.645 |
| OpenAI MMLU (Ar) | 0.412 |
Usage
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = "humain-ai/ALLaM-7B-Instruct-preview"
tok = AutoTokenizer.from_pretrained(base, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(base, trust_remote_code=True, device_map="auto")
model = PeftModel.from_pretrained(model, "ManarAlrabie/arabic-llm-synthetic-500k")
Intended Use & Limitations
Research artifact for studying instruction-data quality vs. quantity in Arabic LLM fine-tuning. Not intended for production. As a single-seed fine-tune of a 7B model, outputs may contain inaccuracies.
Citation
Associated paper is under review; citation will be added upon publication. Until then, please link to this repository.
- Task
- Text Generation
- Library
- peft
- License
- other
- Languages
- ar
- Created
- Feb 3, 2026
- Updated
- Jun 18, 2026