Downloads · 30 days
1.3K
15% of all-time downloads
renketong/Huihui-Qwen3.8-27B-abliterated-NVFP4-GGUF
Huihui-Qwen3.8-27B-abliterated-NVFP4-GGUF is a text generation model from renketong. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
GGUF conversion of huihui-ai / Qwen3.8-27B-abliterated (uncensored) quantized to NVFP4, converted from the source NVFP4 safetensors by sakamakismile.
Downloads · 30 days
1.3K
15% of all-time downloads
All-time downloads
8.3K
Public
Repo size
20.6 GB
Likes
1
Public
Click a slice to open those files.
.gguf20.6 GB · 100%
From the Hugging Face model README
GGUF conversion of huihui-ai / Qwen3.8-27B-abliterated (uncensored) quantized to NVFP4, converted from the source NVFP4 safetensors by sakamakismile.
Upstream lineage:
Qwen/Qwen3.8-27B (Apache-2.0) — base modelhuihui-ai/Huihui-Qwen3.8-27B-abliterated — abliterated (uncensored) fine-tunesakamakismile/Huihui-Qwen3.8-27B-abliterated-NVFP4 — NVFP4 quantized safetensors (compressed-tensors, nvfp4-pack-quantized)convert_hf_to_gguf.py (no re-quantization, no precision loss)| File | Size | Description |
|---|---|---|
Qwen3.8-27B-huihui-NVFP4.gguf | 19.65 GB | Main model. NVFP4 MLP + attention, Q5_K embeddings, BF16 MTP head, 262K native context |
mmproj-huihui.gguf | 931 MB | BF16 vision projector (mmproj) for image input |
qwen35 (Qwen3_5ForConditionalGeneration), 64 layers + 1 MTP layer (nextn_predict_layers=1)# llama.cpp: text-only
llama-server \
-m Qwen3.8-27B-huihui-NVFP4.gguf \
--spec-type draft-mtp \
--spec-draft-n-max 2 \
-c 163840 \
-ngl 999
# llama.cpp: with vision
llama-server \
-m Qwen3.8-27B-huihui-NVFP4.gguf \
--mmproj mmproj-huihui.gguf \
--spec-type draft-mtp \
--spec-draft-n-max 2 \
-c 131072 \
-ngl 999
MTP acceptance rate drives speed; content type drives acceptance rate:
| Content | Draft acceptance | Speed |
|---|---|---|
| Code / JSON | 80–96% | 117–129 t/s |
| Math reasoning | 71% | 114 t/s |
| Chinese / English prose | 37–48% | 88–91 t/s |
Notes:
Lossless repack from compressed-tensors NVFP4 safetensors — no dequantization → requantization round trip:
git clone --depth 1 https://github.com/ggml-org/llama.cpp
pip install -r requirements.txt # torch + numpy + pyyaml + transformers
python convert_hf_to_gguf.py ./Huihui-Qwen3.8-27B-abliterated-NVFP4 \
--outfile Qwen3.8-27B-huihui-NVFP4.gguf --outtype auto
python convert_hf_to_gguf.py ./Huihui-Qwen3.8-27B-abliterated-NVFP4 \
--outfile mmproj-huihui.gguf --mmproj
Conversion is pure CPU (mmap, no VRAM), ~1 minute for 27B on a modern desktop.
Apache-2.0. Base model: Qwen/Qwen3.8-27B. Fine-tune: huihui-ai. Quantization source: sakamakismile. GGUF conversion: renketong.