Downloads · 30 days
10
1% of all-time downloads
RichardErkhov/BramVanroy_-_fietje-2-chat-gguf
BramVanroy_-_fietje-2-chat-gguf is a machine learning model from RichardErkhov. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Downloads · 30 days
10
1% of all-time downloads
All-time downloads
960
Public
Repo size
28.4 GB
Likes
0
Public
Click a slice to open those files.
.gguf33.6 GB · 100%
From the Hugging Face model README
Quantization made by Richard Erkhov.
fietje-2-chat - GGUF
| Name | Quant method | Size |
|---|---|---|
| fietje-2-chat.Q2_K.gguf | Q2_K | 1.03GB |
| fietje-2-chat.Q3_K_S.gguf | Q3_K_S | 1.16GB |
| fietje-2-chat.Q3_K.gguf | Q3_K | 1.33GB |
| fietje-2-chat.Q3_K_M.gguf | Q3_K_M | 1.33GB |
| fietje-2-chat.Q3_K_L.gguf | Q3_K_L | 1.46GB |
| fietje-2-chat.IQ4_XS.gguf | IQ4_XS | 1.43GB |
| fietje-2-chat.Q4_0.gguf | Q4_0 | 1.49GB |
| fietje-2-chat.IQ4_NL.gguf | IQ4_NL | 1.5GB |
| fietje-2-chat.Q4_K_S.gguf | Q4_K_S | 1.5GB |
| fietje-2-chat.Q4_K.gguf | Q4_K | 1.62GB |
| fietje-2-chat.Q4_K_M.gguf | Q4_K_M | 1.62GB |
| fietje-2-chat.Q4_1.gguf | Q4_1 | 1.64GB |
| fietje-2-chat.Q5_0.gguf | Q5_0 | 1.8GB |
| fietje-2-chat.Q5_K_S.gguf | Q5_K_S | 1.8GB |
| fietje-2-chat.Q5_K.gguf | Q5_K | 1.86GB |
| fietje-2-chat.Q5_K_M.gguf | Q5_K_M | 1.86GB |
| fietje-2-chat.Q5_1.gguf | Q5_1 | 1.95GB |
| fietje-2-chat.Q6_K.gguf | Q6_K | 2.12GB |
| fietje-2-chat.Q8_0.gguf | Q8_0 | 2.75GB |
language:
This is the chat version of Fietje, a DPO-tuned (aligned) continuation on the instruct version. Fietje is an adapated version of microsoft/phi-2, tailored to Dutch text generation by training on 28B tokens. It is small and efficient with a size of 2.7 billion parameters while performing almost on par with more powerful Dutch LLMs of twice its size like GEITje 7B Ultra.
A thorough description of the creation and evaluation of Fietje as well as usage examples are available in this Github repository.
The same limitations as phi-2, and LLMs in general, apply here. LLMs hallucinate, make mistakes, and should not be trusted. Use at your own risk!
Fietje 2 Chat was finetuned from the instruct model on the following datasets. Number of training samples per dataset given in brackets, totalling 18,653 samples.
dpo_hq: a cleaned version of BramVanroy/ultra_feedback_dutch (9186)dpo_all: a cleaned version of BramVanroy/orca_dpo_pairs_dutch (9467)A lot of different learning rates, beta, en batch sizes were investigated in search of a converging combination. You can find them all in the W&B runs.
I am thankful to the Flemish Supercomputer Center (VSC) for providing the computational power to accomplish this project. Accounting for waiting for jobs, training a single run took around nine hours on one A100 80GB.
Training was done with the wonderful alignment-handbook, using DeepSpeed as a back-end. Exact training recipes and SLURM script are given in the Github repository.
The following hyperparameters were used during training:
| Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0.2515 | 1.0 | 1166 | 0.2842 | -1.1549 | -3.6363 | 0.8867 | 2.4815 | -657.6813 | -451.3364 | -1.2868 | -1.3528 |
Results for the English Open LLM Leaderboard. For results specific to Dutch, check out ScandEval.
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 10.39 |
| IFEval (0-Shot) | 29.17 |
| BBH (3-Shot) | 17.72 |
| MATH Lvl 5 (4-Shot) | 0.53 |
| GPQA (0-shot) | 0.00 |
| MuSR (0-shot) | 3.20 |
| MMLU-PRO (5-shot) | 11.72 |