Downloads · 30 days
19
15% of all-time downloads
NexesQuants/NeverSleep_MiquMaid-v2-70B-iMat.GGUF
NeverSleep_MiquMaid-v2-70B-iMat.GGUF is a machine learning model from NexesQuants. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Some GGUF Quants with iMatrix for : https://huggingface.co/NeverSleep/MiquMaid-v2-70B
Downloads · 30 days
19
15% of all-time downloads
All-time downloads
125
Public
Repo size
74.9 GB
Likes
0
Public
Click a slice to open those files.
.gguf74.8 GB · 100%
From the Hugging Face model README
Some GGUF Quants with iMatrix for : https://huggingface.co/NeverSleep/MiquMaid-v2-70B
Q3_K_M and IQ3_XXS are here.
For other IQ quants (and possibly better ones than mine), see there : https://huggingface.co/Kooten/MiquMaid-v2-70B-Imatrix-GGUF
Some benchs with LlamaCPP :
NeverSleep_MiquMaid-v2-70B-b2093-iMat-c32_ch1000-Q3_K_M.gguf,-,Hellaswag,83.75,,400,2024-02-07 00:00:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,Nexesenex,
NeverSleep_MiquMaid-v2-70B-b2093-iMat-c32_ch1000-Q3_K_M.gguf,-,Hellaswag,83.1,,1000,2024-02-07 00:00:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,Nexesenex,
NeverSleep_MiquMaid-v2-70B-b2093-iMat-c32_ch1000-Q3_K_M.gguf,-,Arc-Challenge,55.51839465,,299,2024-02-07 05:40:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,Nexesenex,
NeverSleep_MiquMaid-v2-70B-b2093-iMat-c32_ch1000-Q3_K_M.gguf,-,Arc-Easy,77.71929825,,570,2024-02-07 05:40:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,Nexesenex,
NeverSleep_MiquMaid-v2-70B-b2093-iMat-c32_ch1000-Q3_K_M.gguf,-,MMLU,48.56230032,,313,2024-02-07 05:40:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,Nexesenex,
NeverSleep_MiquMaid-v2-70B-b2093-iMat-c32_ch1000-Q3_K_M.gguf,-,Thruthful-QA,41.12607099,,817,2024-02-07 05:40:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,Nexesenex,
NeverSleep_MiquMaid-v2-70B-b2093-iMat-c32_ch1000-Q3_K_M.gguf,-,Winogrande,78.4530,,1267,2024-02-07 05:40:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,Nexesenex,
NeverSleep_MiquMaid-v2-70B-b2093-iMat-c32_ch1000-Q3_K_M.gguf,-,wikitext,4.3333,512,512,2024-02-07 00:00:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,Nexesenex,81
NeverSleep_MiquMaid-v2-70B-b2093-iMat-c32_ch1000-Q3_K_M.gguf,-,wikitext,3.8598,512,512,2024-02-07 00:00:00,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,Nexesenex,655
The Hellaswag might be underestimated by 5-6 points due to recent changes in LlamaCPP.