Downloads · 30 days
65
1% of all-time downloads
Undi95_Miqu-70B-Alpaca-DPO-iMat.GGUF
NexesQuants/Undi95_Miqu-70B-Alpaca-DPO-iMat.GGUF
Undi95_Miqu-70B-Alpaca-DPO-iMat.GGUF is a machine learning model from NexesQuants. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
GGUF Quants with iMatrix for : https://huggingface.co/Undi95/Miqu-70B-Alpaca-DPO
All-time downloads
6K
Public
Repo size
327 GB
Likes
3
Public
Repo makeup
Click a slice to open those files.
.gguf275 GB · 100%
At a glance
- Access
- Public
- Created
- Feb 8, 2024
- Updated
- Mar 11, 2024
- SHA
- 6ba44e6a
Model card
From the Hugging Face model README
GGUF Quants with iMatrix for : https://huggingface.co/Undi95/Miqu-70B-Alpaca-DPO
Q3_K_M to be uploaded shortly.
Q3_K_S, IQ3_XXS, Q2_K, Q2_K_S, IQ2_XS, IQ2_XXS to follow.
LlamaCPP Benchs on the Q3_K_M with iMatrix shared here :
- Undi95_Miqu-70B-Alpaca-DPO-b2101-iMat-c32_ch1000-Q3_K_M.gguf,-,Hellaswag,84.5,,400,2024-02-07 00:00:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,Nexesenex,
- Undi95_Miqu-70B-Alpaca-DPO-b2101-iMat-c32_ch1000-Q3_K_M.gguf,-,Hellaswag,83.6,,1000,2024-02-07 00:00:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,Nexesenex,
- Undi95_Miqu-70B-Alpaca-DPO-b2101-iMat-c32_ch1000-Q3_K_M.gguf,-,Arc-Challenge,58.52842809,,299,2024-02-07 05:40:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,Nexesenex,
- Undi95_Miqu-70B-Alpaca-DPO-b2101-iMat-c32_ch1000-Q3_K_M.gguf,-,Arc-Easy,77.36842105,,570,2024-02-07 05:40:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,Nexesenex,
- Undi95_Miqu-70B-Alpaca-DPO-b2101-iMat-c32_ch1000-Q3_K_M.gguf,-,MMLU,49.84025559,,313,2024-02-07 05:40:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,Nexesenex,
- Undi95_Miqu-70B-Alpaca-DPO-b2101-iMat-c32_ch1000-Q3_K_M.gguf,-,Thruthful-QA,42.83965728,,817,2024-02-07 05:40:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,Nexesenex,
- Undi95_Miqu-70B-Alpaca-DPO-b2101-iMat-c32_ch1000-Q3_K_M.gguf,-,Winogrande,78.7687,,1267,2024-02-07 05:40:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,Nexesenex,
- Undi95_Miqu-70B-Alpaca-DPO-b2101-iMat-c32_ch1000-Q3_K_M.gguf,-,wikitext,4.2963,512,512,2024-02-07 00:00:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,Nexesenex,81
- Undi95_Miqu-70B-Alpaca-DPO-b2101-iMat-c32_ch1000-Q3_K_M.gguf,-,wikitext,3.8397,512,512,2024-02-07 00:00:00,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,Nexesenex,655
LlamaCPP Benchs on a non iMatrix Q3_K_M released by Undi95 :
- Miqu-70B-DPO.q3_k_m.gguf,-,Hellaswag,84.5,400,,2024-02-07 00:00:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,NeverSleep,
- Miqu-70B-DPO.q3_k_m.gguf,-,Hellaswag,83.8,1000,,2024-02-07 00:00:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,NeverSleep,
- Miqu-70B-DPO.q3_k_m.gguf,-,Arc-Challenge,57.85953177,,299,2024-02-07 05:40:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,NeverSleep,
- Miqu-70B-DPO.q3_k_m.gguf,-,Arc-Easy,77.36842105,,570,2024-02-07 05:40:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,NeverSleep,
- Miqu-70B-DPO.q3_k_m.gguf,-,MMLU,50.15974441,,313,2024-02-07 05:40:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,NeverSleep,
- Miqu-70B-DPO.q3_k_m.gguf,-,Thruthful-QA,42.47246022,,817,2024-02-07 05:40:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,NeverSleep,
- Miqu-70B-DPO.q3_k_m.gguf,-,Winogrande,78.7687,,1267,2024-02-07 05:40:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,NeverSleep,
- Miqu-70B-DPO.q3_k_m.gguf,-,wikitext,4.3018,512,512,2024-02-07 00:00:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,NeverSleep,81
- Miqu-70B-DPO.q3_k_m.gguf,-,wikitext,3.8469,512,512,2024-02-07 00:00:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,NeverSleep,655
Quite convincing compared to the original Miqu.. with iMatrix :
- Miqu-1-70b-Requant-b1989-iMat-c32_ch400-Q3_K_M.gguf,-,Arc-Challenge,57.19063545,,299,2024-01-29 05:40:00,,70b,Mistral_Medium,32768,,,GGUF,- Miqudev,Nexesenex,
- Miqu-1-70b-Requant-b1989-iMat-c32_ch400-Q3_K_M.gguf,-,Arc-Easy,77.19298246,,570,2024-01-29 05:40:00,,70b,Mistral_Medium,32768,,,GGUF,- Miqudev,Nexesenex,
- Miqu-1-70b-Requant-b1989-iMat-c32_ch400-Q3_K_M.gguf,-,MMLU,50.15974441,,313,2024-01-29 05:40:00,,70b,Mistral_Medium,32768,,,GGUF,- Miqudev,Nexesenex,
- Miqu-1-70b-Requant-b1989-iMat-c32_ch400-Q3_K_M.gguf,-,Thruthful-QA,41.49326805,,817,2024-01-29 05:40:00,,70b,Mistral_Medium,32768,,,GGUF,- Miqudev,Nexesenex,
- Miqu-1-70b-Requant-b1989-iMat-c32_ch400-Q3_K_M.gguf,-,Winogrande,78.8477,,1267,2024-01-29 05:40:00,,70b,Mistral_Medium,32768,,,GGUF,- Miqudev,Nexesenex,
- Miqu-1-70b-Requant-b1989-iMat-c32_ch400-Q3_K_M.gguf,-,wikitext,4.2957,512,512,2024-01-29 00:00:00,RBF1000000,70b,Mistral_Medium,32768,,,GGUF,- Miqudev,Nexesenex,81
- Miqu-1-70b-Requant-b1989-iMat-c32_ch400-Q3_K_M.gguf,-,wikitext,3.8380,512,512,2024-01-29 00:00:00,RBF1000000,70b,Mistral_Medium,32768,,,GGUF,- Miqudev,Nexesenex,655
The TQA shows a slight bonus, thanks to the DPO training I believe. The slightly bonified ARC benchs (a rare thing on DPO releases!) and the respected perplexity show that the model was not dumbified by the DPO training. In ST, the models performs beautifully.
- Created
- Feb 8, 2024
- Updated
- Mar 11, 2024