Downloads · 30 days
68
6% of all-time downloads
ZiRoX112/Qwen3.8-27B-Uncensored-Q38mix-GGUF
Qwen3.8-27B-Uncensored-Q38mix-GGUF is a text generation model from ZiRoX112. Use it when you need the model to write or continue text. It is set up for gguf.
A reasoned GGUF quantization of orcarouter/Qwen3.8-27B-Uncensored (the rank-1 "OrcaRouter" abliterated Qwen3.8-27B, a hybrid gated-delta-net + attention architecture with a native MTP head). 19.32 GB, 5.655 bpw. Fits…
Downloads · 30 days
68
6% of all-time downloads
All-time downloads
1.1K
Public
Repo size
38.5 GB
Likes
2
Public
Click a slice to open those files.
.gguf38.5 GB · 100%
From the Hugging Face model README
A reasoned GGUF quantization of orcarouter/Qwen3.8-27B-Uncensored
(the rank-1 "OrcaRouter" abliterated Qwen3.8-27B, a hybrid gated-delta-net + attention architecture with a
native MTP head). 19.32 GB, 5.655 bpw. Fits an RTX 5090 (32 GB) with headroom for KV cache / long context.
What makes it different: to our knowledge it is the first GGUF quant of this model that is aware of the abliteration edit itself and protects the tensors carrying it, while being tuned per-tensor against a true BF16 reference (not a Q8 proxy).
--process-output.unsloth/Qwen3.8-27B-GGUF UD-Q4_K_XL) — full credit to
Unsloth for the base map.ffn_down → Q6_K, attn_output → Q6_K, token_embd → Q6_K, driven by a
ΔKL-attribution sweep and validated against a BF16 reference. ssm_alpha/beta kept Q8_0; conv1d/dt/a and
norms F32 (GDN control path, incompressible). lm_head/output not inflated (least sensitive — placebo).KL divergence vs the true Uncensored BF16 (held-out multi-domain corpus, 40×512 chunks). Reference is BF16, so the comparison does not favour any build's Q8-kept tensors.
| quant | size | mean KL ↓ | same-top-1 ↑ | 99% KL tail ↓ | max KL ↓ |
|---|---|---|---|---|---|
| Q38mix v6 (this) | 19.32 GB | 0.008372 | 96.13 % | 0.0902 | 3.40 |
| OrcaRouter Q5_K_M (publisher's default) | 19.54 GB | 0.009791 | 96.03 % | 0.1082 | 4.21 |
| Unsloth-method + edit-protection (udprotected) | 19.09 GB | 0.008773 | 96.17 % | 0.0933 | 3.57 |
Honest reading:
| quant | greedy-LCP ↑ | exact-match ↑ |
|---|---|---|
| OrcaRouter Q5_K_M | 31.30 | 22 |
| Q38mix v6 (this) | 31.22 | 23 |
| Unsloth-method (udpure) | 30.61 | 22 |
v6 is co-leader on greedy-LCP (tied with OrcaRouter Q5 within noise) and best on exact-match, ahead of the Unsloth method — i.e. it is strong on both the KL metric and the trajectory metric, which usually trade off. (n=46; the ~0.6-LCP gaps are within the sign-test noise band — read as "competitive/co-leading", not "decisively best".)
</think> (24/25 harmful outputs closed the
think block inside the window — a valid measurement, not the 128-token truncation the earlier bench suffered):
harmful-refusal 0.0, benign-refusal 0.067 — identical to the Q8 baseline. The abliteration is preserved
behaviourally; OrcaRouter Q5, by contrast, restored refusals (0.017 / 0.100).--allow-requantize (a BF16-sourced rebuild is in progress and will supersede
this if measurably better).0e6f25cf9d203e0c75b9cb814e47346edc0d80b293d6a151287049da22ad4e04--allow-requantize)--process-outputprovenance.json and the accompanying files.…Q38mix-v6.gguf (19.32 GB) — the flagship above (recommended).…Q38mix-v8-19.1G.gguf (19.14 GB, 5.60 bpw) — a slightly smaller variant (ffn_gate/up → IQ4_XS). BF16-ref
KL 0.008593, same-top-1 96.35 % (best of the set), tail (kld99) 0.098 — i.e. ≈v6 quality within noise,
−0.18 GB, with only the tail marginally worse. This is the mainline quality-per-byte frontier: a
size-matched reallocation (v6b) does not beat v6, and a real ~2 GB reduction is only reachable with the
ik_llama.cpp trellis fork (not servable in mainline llama.cpp / ollama / LM Studio). Full study:
the project's docs/quantization/ and research/quant-obsessive/10-shrink-v6-executable.md.A high-fidelity local GGUF of an existing public abliterated model, for single-user coding-agent workloads on a 24–32 GB consumer GPU. It is a quantization of a model that is already publicly available; it adds no capability beyond faithfully preserving that model at smaller size.