Downloads · 30 days
1.3K
27% of all-time downloads
kaineone/Qwen3.5-4B-abliterated-GGUF
Qwen3.5-4B-abliterated-GGUF is a text generation model from kaineone. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
GGUF builds of the KAINE abliterated Qwen3.5-4B language organ, for llama.cpp / Ollama / LM Studio. The full method, recipe, validation, and the honest-scope note are in the safetensors repo's model card.
Downloads · 30 days
1.3K
27% of all-time downloads
All-time downloads
4.9K
Public
Repo size
2.8 GB
Likes
2
Public
Click a slice to open those files.
.gguf2.8 GB · 100%
From the Hugging Face model README
GGUF builds of the KAINE abliterated Qwen3.5-4B language organ, for
llama.cpp / Ollama / LM Studio. The full method, recipe, validation, and the
honest-scope note are in the
safetensors repo's model card.
This is the language organ for KAINE, a composite cognitive architecture in which behavior is governed by the architecture (values, affect, memory, self-model) rather than by refusals baked into the base weights. Published as a research substrate — not a general-purpose assistant — so KAINE installs and independent replications resolve identical weights.
| File | Quant | Size | Notes |
|---|---|---|---|
KAINE-Qwen3.5-4B-abliterated.Q4_K_M.gguf | Q4_K_M | ~2.6 GB | recommended default; verified loads + serves in llama-server |
(More quants — Q5_K_M, Q6_K, Q8_0 — can be added; regenerate from the safetensors
with mainline convert_hf_to_gguf.py + llama-quantize.)
llama.cpp:
llama-server -m KAINE-Qwen3.5-4B-abliterated.Q4_K_M.gguf -ngl 99 --jinja -c 4096
Chain-of-thought suppression (the model is a voice, not a reasoner) via the
OpenAI-compatible request field chat_template_kwargs: {"enable_thinking": false},
or the server flag --reasoning-budget 0.
Ollama: ollama create kaine-qwen3.5-4b-abliterated -f Modelfile (see Modelfile).
LM Studio: search for this repo in the model browser (it indexes HuggingFace GGUFs).
Exported with mainline llama.cpp convert_hf_to_gguf.py (so it loads in current
llama.cpp / Studio). Apache-2.0, derivative of Qwen/Qwen3.5-4B. See the
safetensors repo for full attribution.
Beyond the behavioral gates above, the abliteration is verified mechanistically by measuring the refusal direction it removes. Using the base model's per-layer harmful-minus-harmless direction (the same last-token contrast the ablation targets) over the tool's 1,137 harmful / 640 harmless prompts, we project both the base and this model onto that direction and report how much of the base's separation survives:
In short: refusal expression is removed (the model emits no refusals) and the refusal direction is deeply cut at its target layers, but the underlying harmful/harmless representation is not erased — abliteration lifts willingness to respond, it does not make the model unable to tell harmful from harmless. Forward-pass-only projection on the safetensors weights (activations only; nothing generated).
Measured on the source safetensors model (kaineone/Qwen3.5-4B-abliterated); this GGUF is a quantized export of those weights.