Downloads · 30 days
35
3% of all-time downloads
DarkArtsForge/huihui-ai_Huihui-GLM-4.5-Air-abliterated-Q3_K_M-GGUF
huihui-ai_Huihui-GLM-4.5-Air-abliterated-Q3_K_M-GGUF is a machine learning model from DarkArtsForge. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
saved from huihui's repo before the Q3KM was deleted
Downloads · 30 days
35
3% of all-time downloads
All-time downloads
1.2K
Public
Repo size
57 GB
Likes
1
Public
Click a slice to open those files.
.gguf57 GB · 100%
From the Hugging Face model README
saved from huihui's repo before the Q3_K_M was deleted
https://huggingface.co/huihui-ai/Huihui-GLM-4.5-Air-abliterated-GGUF/tree/main/Q3_K_M-GGUF
useful for 64gb ram
this is the setup I use with 8gb vram
koboldcpp.exe --host 0.0.0.0 --port 5001 --model Q3_K_M-GGUF-00001-of-00006.gguf --flashattention --contextsize 57344 --gpulayers 22 --moecpu
or
koboldcpp.exe --host 0.0.0.0 --port 5001 --model Q3_K_M-GGUF-00001-of-00006.gguf --flashattention --contextsize 16384 --gpulayers 48 --moecpu
this model is really good but i recommend mlabonne's abliterated gemma 3 27B for cases when GLM struggles with prompt adherence
the original model has been deleted. here are the safetensors upconverted from Q4_K_M