GGUF Quants with iMatrix for : https://huggingface.co/jondurbin/bagel-7b-v0.4
Llama Benchs :
Bagel 7b 0.4 (this version) :
- Bagel-7b-v0.4.Q8_0.gguf,-,Hellaswag,80.75,400,2024-02-07 00:00:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,Nexesenex,
- Bagel-7b-v0.4.Q8_0.gguf,-,Hellaswag,79.4,1000,2024-02-07 00:00:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,Nexesenex,
- Bagel-7b-v0.4.Q8_0.gguf,-,Arc-Challenge,51.83946488,,299,2024-02-07 05:40:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,Nexesenex,
- Bagel-7b-v0.4.Q8_0.gguf,-,Arc-Easy,79.29824561,,570,2024-02-07 05:40:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,Nexesenex,
- Bagel-7b-v0.4.Q8_0.gguf,-,MMLU,44.08945687,,313,2024-02-07 05:40:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,Nexesenex,
- Bagel-7b-v0.4.Q8_0.gguf,-,Thruthful-QA,33.53733170,,817,2024-02-07 05:40:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,Nexesenex,
- Bagel-7b-v0.4.Q8_0.gguf,-,Winogrande,76.3220,,1267,2024-02-07 05:40:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,Nexesenex,
- Bagel-7b-v0.4.Q8_0.gguf,-,wikitext,6.1340,512,512,2024-02-07 23:00:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,Nexesenex,
- Bagel-7b-v0.4.Q8_0.gguf,-,wikitext,5.4116,4096,4096,2024-02-07 23:12:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,Nexesenex,
- Bagel-7b-v0.4.Q8_0.gguf,-,wikitext,6.6741,7168,7168,2024-02-07 23:16:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,Nexesenex,
- Bagel-7b-v0.4.Q8_0.gguf,-,wikitext,6.5003,8192,8192,2024-02-07 23:20:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,Nexesenex,
- Bagel-7b-v0.4.Q8_0.gguf,-,wikitext,8.3501,10240,10240,2024-02-07 23:25:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,Nexesenex,
Bagel 7b 0.4 DPO :
- Bagel-dpo-7b-v0.4.Q6_K.gguf,-,Hellaswag,82.75,400,2024-02-07 00:00:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,LoneStriker,
- Bagel-dpo-7b-v0.4.Q6_K.gguf,-,Hellaswag,81.1,1000,2024-02-07 00:00:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,LoneStriker,
- Bagel-dpo-7b-v0.4.Q6_K.gguf,-,Arc-Challenge,53.84615385,,299,2024-02-07 05:40:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,LoneStriker,
- Bagel-dpo-7b-v0.4.Q6_K.gguf,-,Arc-Easy,80.70175439,,570,2024-02-07 05:40:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,LoneStriker,
- Bagel-dpo-7b-v0.4.Q6_K.gguf,-,MMLU,45.36741214,,313,2024-02-07 05:40:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,LoneStriker,
- Bagel-dpo-7b-v0.4.Q6_K.gguf,-,Thruthful-QA,44.55324357,,817,2024-02-07 05:40:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,LoneStriker,
- Bagel-dpo-7b-v0.4.Q6_K.gguf,-,Winogrande,76.4799,,1267,2024-02-07 05:40:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,LoneStriker,
- Bagel-dpo-7b-v0.4.Q6_K.gguf,-,wikitext,6.3245,512,512,2024-02-07 23:30:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,LoneStriker,
- Bagel-dpo-7b-v0.4.Q6_K.gguf,-,wikitext,5.7832,4096,4096,2024-02-07 23:42:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,LoneStriker,
- Bagel-dpo-7b-v0.4.Q6_K.gguf,-,wikitext,8.1732,7168,7168,2024-02-07 23:46:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,LoneStriker,
- Bagel-dpo-7b-v0.4.Q6_K.gguf,-,wikitext,8.0212,8192,8192,2024-02-07 23:50:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,LoneStriker,
- Bagel-dpo-7b-v0.4.Q6_K.gguf,-,wikitext,10.9006,10240,10240,2024-02-07 23:55:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,LoneStriker,
Bagel 7b 0.1 :
- Bagel-7b-v0.1.Q6_K.gguf,-,Hellaswag,86,400,2024-01-27 00:00:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,TheBloke,
- Bagel-7b-v0.1.Q6_K.gguf,-,Hellaswag_Bin,78.25,400,2024-01-27 00:00:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,TheBloke,
- Bagel-7b-v0.1.Q6_K.gguf,-,Arc-Challenge,48.82943144,,299,2024-01-27 05:40:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,TheBloke,
- Bagel-7b-v0.1.Q6_K.gguf,-,Arc-Easy,72.80701754,,570,2024-01-27 05:40:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,TheBloke,
- Bagel-7b-v0.1.Q6_K.gguf,-,MMLU,43.76996805,,313,2024-01-27 05:40:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,TheBloke,
- Bagel-7b-v0.1.Q6_K.gguf,-,Thruthful-QA,31.70134639,,817,2024-01-27 05:40:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,TheBloke,
- Bagel-7b-v0.1.Q6_K.gguf,-,Winogrande,76.4009,,1267,2024-01-27 05:40:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,TheBloke,
- Bagel-7b-v0.1.Q6_K.gguf,-,wikitext,5.82,512,512,2024-01-13 23:00:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,TheBloke,
- Bagel-7b-v0.1.Q6_K.gguf,-,wikitext,5.3503,1024,1024,2024-01-13 23:04:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,TheBloke,
- Bagel-7b-v0.1.Q6_K.gguf,-,wikitext,5.0243,2048,2048,2024-01-13 23:08:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,TheBloke,
- Bagel-7b-v0.1.Q6_K.gguf,-,wikitext,4.9976,4096,4096,2024-01-13 23:12:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,TheBloke,
- Bagel-7b-v0.1.Q6_K.gguf,-,wikitext,5.1589,7168,7168,2024-01-13 23:16:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,TheBloke,
- Bagel-7b-v0.1.Q6_K.gguf,-,wikitext,4.9722,8192,8192,2024-01-13 23:20:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,TheBloke,
- Bagel-7b-v0.1.Q6_K.gguf,-,wikitext,8.999,10240,10240,2024-01-13 23:25:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,TheBloke,
Bagel 7b 0.1 DPO :
- Bagel-dpo-7b-v0.1.Q6_K.gguf,-,Hellaswag,86.25,400,2024-01-27 00:00:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,TheBloke,
- Bagel-dpo-7b-v0.1.Q6_K.gguf,-,Hellaswag_Bin,79.5,400,2024-01-27 00:00:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,TheBloke,
- Bagel-dpo-7b-v0.1.Q6_K.gguf,-,Arc-Challenge,46.48829431,,299,2024-01-27 05:40:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,TheBloke,
- Bagel-dpo-7b-v0.1.Q6_K.gguf,-,Arc-Easy,65.08771930,,570,2024-01-27 05:40:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,TheBloke,
- Bagel-dpo-7b-v0.1.Q6_K.gguf,-,MMLU,44.08945687,,313,2024-01-27 05:40:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,TheBloke,
- Bagel-dpo-7b-v0.1.Q6_K.gguf,-,Thruthful-QA,42.35006120,,817,2024-01-27 05:40:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,TheBloke,
- Bagel-dpo-7b-v0.1.Q6_K.gguf,-,Winogrande,75.8485,,1267,2024-01-27 05:40:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,TheBloke,
- Bagel-dpo-7b-v0.1.Q6_K.gguf,-,wikitext,5.8674,512,512,2024-01-13 23:30:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,TheBloke,
- Bagel-dpo-7b-v0.1.Q6_K.gguf,-,wikitext,5.3889,1024,1024,2024-01-13 23:34:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,TheBloke,
- Bagel-dpo-7b-v0.1.Q6_K.gguf,-,wikitext,5.0578,2048,2048,2024-01-13 23:38:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,TheBloke,
- Bagel-dpo-7b-v0.1.Q6_K.gguf,-,wikitext,5.012,4096,4096,2024-01-13 23:42:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,TheBloke,
- Bagel-dpo-7b-v0.1.Q6_K.gguf,-,wikitext,5.1349,7168,7168,2024-01-13 23:46:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,TheBloke,
- Bagel-dpo-7b-v0.1.Q6_K.gguf,-,wikitext,4.9318,8192,8192,2024-01-13 23:50:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,TheBloke,
- Bagel-dpo-7b-v0.1.Q6_K.gguf,-,wikitext,8.3897,10240,10240,2024-01-13 23:55:00,,07b,Mistral_7b_v02,8192,,,GGUF,JonDurbin,TheBloke,