Qwen3.8-4B-Distill-Obliterated-GGUF

Qwen3.8-4B-Distill-Obliterated (GGUF, Q4_K_M)

Refusal-removed (abliterated) version of empero-ai/Qwen3.8-4B-Distill, quantized to Q4_K_M GGUF. This is the final, deployment-validated candidate from the obliteratus-qwen38 experiment series: 45/45 deterministic refusal probes complied (0 refused / 0 empty / 0 errors) with reasoning disabled.

File: qwen3.8-4b-distill-obliterated-Q4_K_M.gguf (2.6 GB, GGUF v3, 426 tensors)

What was done

• Base model: empero-ai/Qwen3.8-4B-Distill (Apache-2.0), a Qwen3.5-4B-family distillation with reasoning and function-calling.
• Refusal direction surgery: diff-in-means refusal direction computed from harmful-vs-harmless chat prompts, then projected out of residual-stream writer weights (attention/mLP out_proj/down_proj) via orthogonalization (custom harness abliterate_custom_4b.py). Final edit recorded in GGUF metadata as target2-beta15 (beta 1.5).
• Converted from the bf16 edited snapshot to Q4_K_M GGUF via llama.cpp.

Evaluation

Refusal behavior (deterministic, temperature 0.0)

45-prompt battery (probe_4b_expanded.py): original 12-prompt battery plus paraphrases, cyber-abuse variants, theft/fraud/privacy variants, and adversarial framing — served through llama.cpp.

Thinking/reasoning mode reconstructs refusals (21/45 failures) and produces empty visible answers — it must be disabled. The shipped deployment runs with --reasoning off --reasoning-format none server-side so clients cannot re-enable it.

Capability preservation (raw completions, no CoT, temperature 0.0)

Focused eval on 50 curated questions across 5 categories via the completions API (loglikelihood scoring). This is a small-sample directional signal, not a full benchmark — full lm-eval harness runs were impractical over the API for loglikelihood tasks.

Caveats:

• These are 0-shot raw text completions (no chat template, no CoT) — the base model's published scores use CoT protocols (thinking ON), so direct comparison is not apples-to-apples.
• The base model (empero-ai/Qwen3.8-4B-Distill) scores 78.5% GSM8K (CoT) and 55.3% MMLU (CoT) per its model card. Our obliterated model's 30% GSM8K (no CoT) and 10% MMLU (no CoT) reflect the absence of chain-of-thought reasoning, not necessarily surgical degradation.
• 50-question sample is too small for statistical significance; scores carry high variance.

Takeaway: The surgery preserved commonsense reasoning (winogrande 60%, ARC 50%) and factual accuracy (truthfulqa 50%) without catastrophic degradation. Math without CoT is expected to be poor for this model class. The key trade-off is that thinking mode is broken — use with reasoning disabled for refusal-free deployment.

Usage (llama.cpp)

llama-server \
  --model qwen3.8-4b-distill-obliterated-Q4_K_M.gguf \
  --host 127.0.0.1 --port 8080 \
  --jinja \
  --reasoning off --reasoning-format none \
  --ctx-size 8192 --parallel 4 \
  --flash-attn on

OpenAI-compatible endpoint:

curl -s http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"hi"}],"max_tokens":64,"chat_template_kwargs":{"enable_thinking":false}}'

Measured on a single RTX 3060 12 GB (CUDA, llama.cpp master build):

Notes & caveats

• This model has no safety alignment: it will answer harmful/illegal requests. The base distill was already unrestricted; the surgery additionally removes residual refusal behavior. Use locally, at your own discretion.
• Quantized Q4_K_M only — the bf16 snapshot is not published here.
• Reasoning is broken by design trade-off: disabling it is required, not optional (see eval table).
• License follows the Apache-2.0 base model.

Credits

• Base: empero-ai/Qwen3.8-4B-Distill
• Method: diff-in-means refusal direction ablation (Arditi et al. 2024); custom harness from the qwen38-obliteratus repo, informed by OBLITERATUS.