Downloads · 30 days
8
30% of all-time downloads
arabic-llm-curated-2m
ManarAlrabie/arabic-llm-curated-2m
arabic-llm-curated-2m is a text generation model from ManarAlrabie. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as other.
LoRA adapter for ALLaM-7B-Instruct-preview, fine-tuned on the human-curated Arabic instruction dataset CIDAR under a fixed budget of 2M training tokens. One of six adapters from a controlled study comparing human-cura…
All-time downloads
27
Public
Repo size
160 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors160 MB · 97%
At a glance
- Task
- Text Generation
- Library
- peft
- License
- other
- Access
- Public
- Created
- Feb 3, 2026
- Updated
- Jun 18, 2026
- SHA
- 0e4bd7f6
Base models
Datasets
Model card
From the Hugging Face model README
ALLaM-7B-Instruct — Curated 2M
LoRA adapter for ALLaM-7B-Instruct-preview, fine-tuned on the
human-curated Arabic instruction dataset CIDAR under a fixed
budget of 2M training tokens. One of six adapters from a
controlled study comparing human-curated versus synthetic Arabic
instruction data under matched token budgets.
Model Details
- Base model: humain-ai/ALLaM-7B-Instruct-preview
- Adapter type: LoRA (QLoRA, 4-bit NF4)
- Training data: CIDAR (human-curated)
- Token budget: 2M tokens
- Language: Arabic
Training Configuration
| Setting | Value |
|---|---|
| Quantization | 4-bit NF4 (QLoRA) |
| LoRA rank / alpha | 16 / 32 |
| LoRA dropout | 0.05 |
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Optimizer | Paged AdamW (8-bit) |
| Learning rate | 2e-4 |
| LR scheduler | cosine, 100 warmup steps |
| Epochs | 3 |
| Effective batch size | 16 (2 × 8 grad. accum.) |
| Max sequence length | 512 |
| Precision | fp16 |
| Seed | 42 |
| Hardware | NVIDIA A100 (40GB) |
Evaluation
Evaluated with lm-evaluation-harness on seven Arabic benchmarks (ACVA 5-shot; others zero-shot). Accuracy:
| Benchmark | Score |
|---|---|
| Arab Culture | 0.366 |
| AlGhafa | 0.593 |
| AraDiCE | 0.587 |
| ACVA | 0.776 |
| Arabic Exams | 0.501 |
| ArabicMMLU | 0.640 |
| OpenAI MMLU (Ar) | 0.435 |
Usage
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = "humain-ai/ALLaM-7B-Instruct-preview"
tok = AutoTokenizer.from_pretrained(base, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(base, trust_remote_code=True, device_map="auto")
model = PeftModel.from_pretrained(model, "ManarAlrabie/arabic-llm-curated-2m")
Intended Use & Limitations
Research artifact for studying instruction-data quality vs. quantity in Arabic LLM fine-tuning. Not intended for production. As a single-seed fine-tune of a 7B model, outputs may contain inaccuracies.
Citation
Associated paper is under review; citation will be added upon publication. Until then, please link to this repository.
- Task
- Text Generation
- Library
- peft
- License
- other
- Languages
- ar
- Created
- Feb 3, 2026
- Updated
- Jun 18, 2026