Downloads · 30 days
399
1% of all-time downloads
Lewdiculous/SOVL_Llama3_8B-GGUF-IQ-Imatrix
SOVL_Llama3_8B-GGUF-IQ-Imatrix is a machine learning model from Lewdiculous. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
[!TIP] My upload speeds have been cooked and unstable lately. <br Realistically I'd need to move to get a better provider. <br If you want and you are able to, you can support various endeavors here (Ko-fi). <br I apo…
Downloads · 30 days
399
1% of all-time downloads
All-time downloads
30.4K
Public
Repo size
137 GB
Likes
25
Public
Click a slice to open those files.
.gguf68.3 GB · 100%
From the Hugging Face model README
[!TIP] My upload speeds have been cooked and unstable lately. <br> Realistically I'd need to move to get a better provider. <br> If you want and you are able to, you can support various endeavors here (Ko-fi). <br> I apologize for disrupting your experience.
GGUF-IQ-Imatrix quants for @jeiku's ResplendentAI/SOVL_Llama3_8B. <br> Give them some love!
[!IMPORTANT]
Updated! These quants have been redone with the fixes from llama.cpp/pull/6920 in mind. <br> Use KoboldCpp version 1.64 or higher.
[!NOTE] Well...! <br> Turns out it was not just a hallucination and this model actually is pretty cool so give it a chance! <br> For 8GB VRAM GPUs, I recommend the Q4_K_M-imat quant for up to 12288 context sizes.
[!WARNING] Use the provided presets. <br> Compatible SillyTavern presets here (simple) or here (Virt's roleplay). Use the latest version of KoboldCpp.
