Downloads · 30 days
956
19% of all-time downloads
ReBatch_-_Llama-3-8B-dutch-gguf
RichardErkhov/ReBatch_-_Llama-3-8B-dutch-gguf
ReBatch_-_Llama-3-8B-dutch-gguf is a machine learning model from RichardErkhov. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
All-time downloads
5.1K
Public
Repo size
92.9 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf108 GB · 100%
At a glance
- Access
- Public
- Created
- Aug 19, 2024
- Updated
- Aug 19, 2024
- SHA
- fc292298
Model card
From the Hugging Face model README
Quantization made by Richard Erkhov.
Llama-3-8B-dutch - GGUF
- Model creator: https://huggingface.co/ReBatch/
- Original model: https://huggingface.co/ReBatch/Llama-3-8B-dutch/
| Name | Quant method | Size |
|---|---|---|
| Llama-3-8B-dutch.Q2_K.gguf | Q2_K | 2.96GB |
| Llama-3-8B-dutch.IQ3_XS.gguf | IQ3_XS | 3.28GB |
| Llama-3-8B-dutch.IQ3_S.gguf | IQ3_S | 3.43GB |
| Llama-3-8B-dutch.Q3_K_S.gguf | Q3_K_S | 3.41GB |
| Llama-3-8B-dutch.IQ3_M.gguf | IQ3_M | 3.52GB |
| Llama-3-8B-dutch.Q3_K.gguf | Q3_K | 3.74GB |
| Llama-3-8B-dutch.Q3_K_M.gguf | Q3_K_M | 3.74GB |
| Llama-3-8B-dutch.Q3_K_L.gguf | Q3_K_L | 4.03GB |
| Llama-3-8B-dutch.IQ4_XS.gguf | IQ4_XS | 4.18GB |
| Llama-3-8B-dutch.Q4_0.gguf | Q4_0 | 4.34GB |
| Llama-3-8B-dutch.IQ4_NL.gguf | IQ4_NL | 4.38GB |
| Llama-3-8B-dutch.Q4_K_S.gguf | Q4_K_S | 4.37GB |
| Llama-3-8B-dutch.Q4_K.gguf | Q4_K | 4.58GB |
| Llama-3-8B-dutch.Q4_K_M.gguf | Q4_K_M | 4.58GB |
| Llama-3-8B-dutch.Q4_1.gguf | Q4_1 | 4.78GB |
| Llama-3-8B-dutch.Q5_0.gguf | Q5_0 | 5.21GB |
| Llama-3-8B-dutch.Q5_K_S.gguf | Q5_K_S | 5.21GB |
| Llama-3-8B-dutch.Q5_K.gguf | Q5_K | 5.34GB |
| Llama-3-8B-dutch.Q5_K_M.gguf | Q5_K_M | 5.34GB |
| Llama-3-8B-dutch.Q5_1.gguf | Q5_1 | 5.65GB |
| Llama-3-8B-dutch.Q6_K.gguf | Q6_K | 6.14GB |
| Llama-3-8B-dutch.Q8_0.gguf | Q8_0 | 7.95GB |
Original model description:
license: llama3 base_model: meta-llama/Meta-Llama-3-8B tags:
- ORPO
- llama 3 8B
- conversational datasets:
- BramVanroy/ultra_feedback_dutch model-index:
- name: ReBatch/Llama-3-8B-dutch results: [] language:
- nl pipeline_tag: text-generation
<p align="center" style="margin:0;padding:0"> <img src="llama3-8b-dutch-banner.jpeg" alt="Llama 3 dutch banner" width="400" height="400"/> </p> <div style="margin:auto; text-align:center"> <h1 style="margin-bottom: 0">Llama 3 8B - Dutch</h1> <em>A conversational model for Dutch, based on Llama 3 8B</em> <p><em><a href="https://huggingface.co/spaces/ReBatch/Llama-3-Dutch">Try chatting with the model!</a></em></p> </div>
This model is a QLORA and ORPO fine-tuned version of meta-llama/Meta-Llama-3-8B on the synthetic feedback dataset BramVanroy/ultra_feedback_dutch
Model description
This model is a Dutch chat model, originally developed from Llama 3 8B and further refined through a feedback dataset with ORPO and trained on BramVanroy/ultra_feedback_dutch
Intended uses & limitations
Although the model has been aligned with gpt-4-turbo output, which has strong content filters, the model could still generate wrong, misleading, and potentially even offensive content. Use at your own risk.
Training procedure
The model was trained in bfloat16 with QLORA with flash attention 2 on one GPU - H100 80GB SXM5 for around 24 hours on RunPod.
Evaluation Results
The model was evaluated using scandeval
The model showed mixed results across different benchmarks; it exhibited slight improvements on some while experiencing a decrease in scores on others. This occurred despite being trained on only 200,000 samples for a single epoch. We are curious to see whether its performance could be enhanced by training with more data or additional epochs.
| Model | conll_nl | dutch_social | scala_nl | squad_nl | wiki_lingua_nl | mmlu_nl | hellaswag_nl |
|---|---|---|---|---|---|---|---|
| meta-llama/Meta-Llama-3-8B-Instruct | 68.72 | 14.67 | 32.91 | 45.36 | 67.62 | 36.18 | 33.91 |
| ReBatch/Llama-3-8B-dutch | 58.85 | 11.14 | 15.58 | 59.96 | 64.51 | 36.27 | 28.34 |
| meta-llama/Meta-Llama-3-8B | 62.26 | 10.45 | 30.3 | 62.99 | 65.17 | 36.38 | 28.33 |
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 8e-06
- train_batch_size: 2
- eval_batch_size: 2
- num_devices: 1
- gradient_accumulation_steps: 4
- optimizer: paged_adamw_8bit
- lr_scheduler_type: linear
- warmup_steps: 10
- num_epochs: 1.0
- r: 16
- lora_alpha: 32
- lora_dropout: 0.05
- Created
- Aug 19, 2024
- Updated
- Aug 19, 2024