Qwen3.8-27B-Heretic-GGUF

Qwen3.8-27B Heretic GGUF

GGUF builds of darrellbest/Qwen3.8-27B-Heretic: Qwen3.8-27B with its refusal behaviour removed (0/100 refusals vs 98/100 for the original, KL divergence 0.0465).

Files

The three model files are text-only: this is a vision-language model, and llama.cpp keeps the vision encoder in the separate mmproj file. Add it only if you want to send images.

Ollama

ollama create qwen3.8-27b-heretic -f Modelfile     # Modelfile is in this repo
ollama run qwen3.8-27b-heretic

The included Modelfile sets RENDERER qwen3.8 and PARSER qwen3.5, Ollama's built-in support for this architecture. They matter: without them the model's reasoning text ends up inside the answer, and generation was measured about half as fast (9.5 vs 18.8 tokens/s on an RTX PRO 6000 with Q4_K_M).

For images, add a second FROM line pointing at the mmproj file.

llama.cpp

llama-cli -m Qwen3.8-27B-Heretic-Q8_0.gguf --jinja
llama-cli -m Qwen3.8-27B-Heretic-Q8_0.gguf --mmproj Qwen3.8-27B-Heretic-mmproj-F16.gguf --jinja   # with images

Checks

Both quantizations were imported into Ollama and run before publishing: each answered a normal question coherently, and neither produced a refusal on a prompt the original model refuses, so the ablation survives quantization. Speed on an RTX PRO 6000: 11.6 tokens/s (Q8_0) and 9.5 (Q4_K_M) as plain GGUF imports, 18.8 with the Modelfile above; the stock qwen3.8 from Ollama's registry runs 12.4 on the same machine, so these are in line with the runtime rather than slowed by the conversion.

Converted with llama.cpp bd4f514 (convert_hf_to_gguf.py, then llama-quantize).

Ablation

Method, parameters and measurements are on the parent card.

Reduced safety guardrails by design. You are responsible for what you do with it.

The family