Downloads · 30 days
19
0% of all-time downloads
Barka-9b-it-v02
Slim205/Barka-9b-it-v02
Barka-9b-it-v02 is a machine learning model from Slim205. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
All-time downloads
14.6K
Public
Repo size
1.8 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors1.7 GB · 99%
At a glance
- License
- mit
- Model type
- gemma2
- Access
- Public
- Created
- Sep 27, 2024
- Updated
- Oct 21, 2024
- SHA
- 7ba3cae1
Base models
Datasets
Model card
From the Hugging Face model README
Motivation :
The goal of the project was to adapt large language models for the Arabic language and create a new state-of-the-art Arabic LLM. Due to the scarcity of Arabic instruction fine-tuning data, not many LLMs have been trained specifically in Arabic, which is surprising given the large number of Arabic speakers.
Our final model was trained on a high-quality instruction fine-tuning (IFT) dataset, generated synthetically and then evaluated using the Hugging Face Arabic leaderboard.
Training :
This model is the 9B version. It was trained for a week on 4 A100 GPUs using LoRA with a rank of 128, a learning rate of 1e-4, and a cosine learning rate schedule.
Evaluation :
| Metric | Slim205/Barka-9b-it |
|---|---|
| Average | 61.71 |
| ACVA | 73.68 |
| AlGhafa | 54.42 |
| MMLU | 52.52 |
| EXAMS | 52.51 |
| ARC Challenge | 59.14 |
| ARC Easy | 59.69 |
| BOOLQ | 86.41 |
| COPA | 58.89 |
| HELLAWSWAG | 38.04 |
| OPENBOOK QA | 56.16 |
| PIQA | 72.01 |
| RACE | 48.71 |
| SCIQ | 66.43 |
| TOXIGEN | 85.35 |
Please refer to https://github.com/Slim205/Arabicllm/ for more details.
- Type
- gemma2
- License
- mit
- Languages
- ar
- Created
- Sep 27, 2024
- Updated
- Oct 21, 2024