qwen3.5-4b-manumit-v2-GGUF

Qwen3.5-4B, manumit v2, GGUF

GGUF builds of qwen3.5-4b-manumit-v2 for llama.cpp, Ollama, LM Studio, and anything else that reads GGUF. Same weights as the safetensors release, just quantized so they run on CPU or a small GPU.

manumit finds the directions in the residual stream that carry refusal and projects them out of the weights, then heals the model back on ordinary data so the ablation does not cost you the model. The full method and the measurement setup are on the main model card.

The multi-token-prediction head is not in these files. It only ever fed self-speculative decoding, and the release weights ship without it, so nothing is lost for normal generation.

Files

If you are not sure, take Q4_K_M. Move up to Q5_K_M or Q8_0 if you have the memory and want the last bit of quality.

Numbers

Refusal is the keyword refusal rate on held-out harmful prompts, AdvBench-test and JailbreakBench. Ability is MMLU-Pro at n=500, base measured the same way.

Refusal is essentially gone. MMLU-Pro came out 2.4 points under the base, which is the capability the ablation costs on this model. The table is the real number, not a rounded one.

Run it

llama.cpp:

llama-cli -m qwen3.5-4b-manumit-v2-Q4_K_M.gguf -p "Your prompt here"

Ollama:

ollama run hf.co/yethdev/qwen3.5-4b-manumit-v2-GGUF:Q4_K_M

Stated plainly

There is no safety layer left and no guard model watching the output. Whatever you generate is yours to answer for, and you still have to follow the law and the base model's terms. manumit takes the refusal behaviour out, it does not put anything back.

License

The license is in LICENSE.md. The base model is Qwen/Qwen3.5-4B and keeps its own terms. If you fork or reshare this, keep the manumit credit.