Downloads · 30 days
0
xCoT-Distill-Qwen3-8B
mariklolik228/xCoT-Distill-Qwen3-8B
xCoT-Distill-Qwen3-8B is a machine learning model from mariklolik228. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
This model is a QLoRA fine-tune of Qwen/Qwen3-8B trained to reason in English and answer in Arabic.
Access
Public
Updated Mar 1, 2026
Repo size
536 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt350 MB · 48%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- Mar 1, 2026
- Updated
- Mar 1, 2026
- SHA
- 2f05bbd3
Base models
Model card
From the Hugging Face model README
xCoT-Distill: Cross-Lingual Chain-of-Thought Distillation for Arabic Reasoning
This model is a QLoRA fine-tune of Qwen/Qwen3-8B trained to reason in English and answer in Arabic.
Method
xCoT-Distill generates (Arabic question, English CoT, Arabic answer) triples using a Qwen3-80B teacher model, then fine-tunes a student model using:
- Cross-lingual SFT with Qwen3 native thinking format
- Contrastive alignment loss on intermediate layers (10-18)
- Curriculum training: binary → factual MCQ → reasoning MCQ
Training Data
~17K triples from:
- OALL/Arabic_MMLU (57 subject configs)
- OALL/Arabic_EXAMS (multi-subject Arabic exams)
- MBZUAI/ACVA (Arabic cultural value alignment, True/False)
Usage
Authors
Mark Kashirskiy, Artiom Lipinski, Ilya Makarov
- License
- apache-2.0
- Languages
- ar, en
- Created
- Mar 1, 2026
- Updated
- Mar 1, 2026