Downloads · 30 days
8
17% of all-time downloads
arabic-llm-synthetic-1m
ManarAlrabie/arabic-llm-synthetic-1m
arabic-llm-synthetic-1m is a text generation model from ManarAlrabie. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as other.
LoRA adapter for ALLaM-7B-Instruct-preview, fine-tuned on the synthetically generated portion of InstAr-500k under a fixed budget of 1M training tokens. One of six adapters from a controlled study comparing human-cura…
All-time downloads
47
Public
Repo size
160 MB
Likes
1
Trending 1
Repo makeup
Click a slice to open those files.
.safetensors160 MB · 97%
At a glance
- Task
- Text Generation
- Library
- peft
- License
- other
- Access
- Public
- Created
- Feb 4, 2026
- Updated
- Jun 18, 2026
- SHA
- 765563f2
Base models
Datasets
Model card
From the Hugging Face model README
ALLaM-7B-Instruct — Synthetic 1M
LoRA adapter for ALLaM-7B-Instruct-preview, fine-tuned on the
synthetically generated portion of InstAr-500k under a fixed
budget of 1M training tokens. One of six adapters from a
controlled study comparing human-curated versus synthetic Arabic
instruction data under matched token budgets.
Model Details
- Base model: humain-ai/ALLaM-7B-Instruct-preview
- Adapter type: LoRA (QLoRA, 4-bit NF4)
- Training data: InstAr-500k (synthetically generated portion; human-crafted sources, including CIDAR, excluded)
- Token budget: 1M tokens
- Language: Arabic
Training Configuration
| Setting | Value |
|---|---|
| Quantization | 4-bit NF4 (QLoRA) |
| LoRA rank / alpha | 16 / 32 |
| LoRA dropout | 0.05 |
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Optimizer | Paged AdamW (8-bit) |
| Learning rate | 2e-4 |
| LR scheduler | cosine, 100 warmup steps |
| Epochs | 3 |
| Effective batch size | 16 (2 × 8 grad. accum.) |
| Max sequence length | 512 |
| Precision | fp16 |
| Seed | 42 |
| Hardware | NVIDIA A100 (40GB) |
Evaluation
Evaluated with lm-evaluation-harness on seven Arabic benchmarks (ACVA 5-shot; others zero-shot). Accuracy:
| Benchmark | Score |
|---|---|
| Arab Culture | 0.355 |
| AlGhafa | 0.538 |
| AraDiCE | 0.584 |
| ACVA | 0.753 |
| Arabic Exams | 0.497 |
| ArabicMMLU | 0.640 |
| OpenAI MMLU (Ar) | 0.378 |
Usage
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = "humain-ai/ALLaM-7B-Instruct-preview"
tok = AutoTokenizer.from_pretrained(base, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(base, trust_remote_code=True, device_map="auto")
model = PeftModel.from_pretrained(model, "ManarAlrabie/arabic-llm-synthetic-1m")
Intended Use & Limitations
Research artifact for studying instruction-data quality vs. quantity in Arabic LLM fine-tuning. Not intended for production. As a single-seed fine-tune of a 7B model, outputs may contain inaccuracies.
Citation
Associated paper is under review; citation will be added upon publication. Until then, please link to this repository.
- Task
- Text Generation
- Library
- peft
- License
- other
- Languages
- ar
- Created
- Feb 4, 2026
- Updated
- Jun 18, 2026