Downloads · 30 days
0
AraSwap-Qwen3-8B-Adapters
mariklolik228/AraSwap-Qwen3-8B-Adapters
AraSwap-Qwen3-8B-Adapters is a machine learning model from mariklolik228. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Three separately trained LoRA experts for Qwen3-8B, composed via layer-selective merging.
Access
Public
Updated Mar 2, 2026
Repo size
1.7 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors1.7 GB · 100%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- Mar 2, 2026
- Updated
- Mar 2, 2026
- SHA
- 58a9057f
Base models
Model card
From the Hugging Face model README
AraSwap: Layer-Level Capability Composition for Arabic LLM Adaptation
Three separately trained LoRA experts for Qwen3-8B, composed via layer-selective merging.
Adapters
| Adapter | Data | LoRA r | Steps |
|---|---|---|---|
arabic_expert/ | Arabic Wikipedia + Arabic MMLU | 64 | 1000 |
math_expert/ | GSM8K + MATH (English) | 64 | 1000 |
cultural_expert/ | ACVA + Arabic cultural subjects | 32 | 1000 |
Method
Frobenius norm analysis of each expert's weight delta identifies layer specialization. Under a strict 15% dominance criterion, 4 layers are selected for swapping:
- Layers 1, 9, 16: Arabic Language Expert
- Layer 23: Arabic Cultural Expert
- Remaining 32 layers: base model (reasoning preserved)
Results
AraSwap merged OALL avg: 62.1% (4-bit, 0-shot)
Key finding: AraTrust 85.6% > baseline 85.2% (+0.4pp).
Usage
from peft import PeftModel
from transformers import AutoModelForCausalLM
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-8B")
model = PeftModel.from_pretrained(base, "mariklolik228/AraSwap-Qwen3-8B-Adapters", subfolder="arabic_expert")
Citation
@article{kashirskiy2026araswap,
title={AraSwap: Layer-Level Capability Composition for Arabic LLM Adaptation},
author={Kashirskiy, Mark and Lipinski, Artiom and Makarov, Ilya},
year={2026}
}
- License
- apache-2.0
- Languages
- ar, en
- Created
- Mar 2, 2026
- Updated
- Mar 2, 2026