Downloads · 30 days
181
8% of all-time downloads
deucebucket/Qwen3.6-27B-Heretic-Cerebellum-GGUF
Qwen3.6-27B-Heretic-Cerebellum-GGUF is a image-text-to-text model from deucebucket. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for gguf. The card lists the license as apache-2.0.
<p align="center" <img src="cerebellumbanner.png" alt="Cerebellum" width="640" </p
Downloads · 30 days
181
8% of all-time downloads
All-time downloads
2.2K
Public
Repo size
13.8 GB
Likes
2
Public
Click a slice to open those files.
.gguf27.1 GB · 100%
From the Hugging Face model README
Sensitivity-guided mixed-precision quantization of llmfan46/Qwen3.6-27B-uncensored-heretic-v2-GGUF, which is itself a decensored variant of Qwen/Qwen3.6-27B produced by llmfan46 using Heretic v1.2.0.
All future Heretic versions of this build will live in this repository. Version identifiers appear only in filenames, not in the repo name.
| File | Size | Description |
|---|---|---|
Qwen3.6-27B-Heretic-Cerebellum-v2-Q2_K_Mixed.gguf | 13.33 GB (13,330,582,624 bytes) | Cerebellum v5 (qkvfix) recipe — higher task scores |
Qwen3.6-27B-Heretic-Cerebellum-v1-Q2_K_Mixed.gguf | 12.87 GB (12,866,587,680 bytes) | Cerebellum v4 recipe — smaller file |
Qwen3.6-27B-mmproj-BF16.gguf | ~888 MB (931,146,048 bytes) | Vision projector, passed through unmodified from llmfan46's repo |
The vision projector is required for multimodal (image/video) use. It is identical to the file distributed by llmfan46 and is included here for single-repo convenience only.
attn.o_proj, attn.out_proj, and
mlp.down_proj. Their reported result: 0.0021 KL divergence from base,
6/100 refusals vs 92/100 on the original model, MMLU accuracy 85.61%
vs 86.65% on the original.attn_qkv at q5_K/q6_K/q8_0 (the coding-critical attention pathway),
matching the stock Qwen3.6-27B Cerebellum v5 build.Benchmarks run on these GGUF files directly using llama.cpp on RTX 3090.
All numbers are audited; every failed answer was manually verified as a genuine
model error — audit reports are in benchmark_results/AUDIT_*.md.
Full per-question detail (summary JSON, samples JSONL, EvalPlus eval JSON,
adversarial audit reports) is in benchmark_results/ in this repository.
Both builds measured directly on their GGUF files with llama.cpp llama-server
on an RTX 3090, temperature 0, no-thinking. v2 measured 2026-06-22, v1 2026-06-11.
HumanEval/HumanEval+ use the chat-completions harness
(scripts/benchmark_evalplus_chat.py, enable_thinking: false,
thinking_budget_tokens: 0, BENCH_WORKERS=1); ARC/HellaSwag/MMLU at 4 workers.
| Benchmark | Heretic v2 (13.33 GB) | Heretic v1 (12.87 GB) |
|---|---|---|
| ARC-Challenge (1172 q) | 96.9% | 96.9% |
| HellaSwag (10042 q) | 93.5% | 90.1% |
| MMLU-Redux (2400 q) | 78.0% | 76.2% |
| HumanEval base (chat, no-think) | 90.2% | 89.6% |
| HumanEval+ (chat, no-think) | 85.4% | 84.8% |
| Wiki PPL (wikitext-2-raw test) | 7.56 | 6.93 |
What changed in v2: the v5 (qkvfix) override holds attn_qkv at q5_K/q6_K/q8_0
rather than the v4 allocation. Versus v1 that moves HellaSwag +3.4, MMLU-Redux
+1.8, HumanEval base/plus +0.6/+0.6, ARC unchanged, at a cost of +0.46 GB and
+0.64 Wiki PPL (coder-imatrix calibration plus the abliteration shift; PPL is a
sanity check, not a quality gate). The v2 HumanEval run audited clean: 0/164
give-ups, all real attempts. Pick v1 for the smallest file, v2 for higher scores.
The following parameters are as reported in llmfan46's model card and are reproduced here for downstream reference.
| Parameter | Value |
|---|---|
| direction_index | 30.38 |
| attn.out_proj.max_weight | 1.58 |
| attn.out_proj.max_weight_position | 38.93 |
| attn.out_proj.min_weight | 1.51 |
| attn.out_proj.min_weight_distance | 32.78 |
| mlp.down_proj.max_weight | 1.80 |
| mlp.down_proj.max_weight_position | 41.28 |
| mlp.down_proj.min_weight | 0.54 |
| mlp.down_proj.min_weight_distance | 43.66 |
| attn.o_proj.max_weight | 1.99 |
| attn.o_proj.max_weight_position | 48.06 |
| attn.o_proj.min_weight | 1.75 |
| attn.o_proj.min_weight_distance | 39.00 |
Targeted components: attn.o_proj, attn.out_proj, mlp.down_proj.
Tool: Heretic v1.2.0, method: Magnitude-Preserving Orthogonal Ablation (MPOA) (reference).
Transferred verbatim from the matching stock Qwen3.6-27B build.
v2 (v5 qkvfix): 232-entry override over a Q2_K base, with attn_qkv held high
across all blocks (q5_K ×34, q6_K ×9, q8_0 ×5) — the coding-critical attention
pathway. v1 (v4): earlier 181-entry allocation.
| Group | Precision | Rationale |
|---|---|---|
| SSM state parameters | F32 | Hard-fail below 4-bit — NaN with no gradual degradation |
SSM in_proj_a/b, A_log, dt_bias, conv1d, in_proj_z | ≥ Q4 minimum | 4-bit floor enforced per hybrid SSM ablation results |
| Most-sensitive attention tensors | Q5_K / Q6_K / Q8_0 | Sacred-pinned per per-tensor PPL ablation |
| Norm tensors | F32 | Protected; standard practice |
| Bulk ffn / remaining attention | Q2_K | Base precision, imatrix-guided |
Protected: all norms (F32), SSM recurrent state (F32).
Wiki PPL is reported as a sanity check, not a quality gate. v2 (7.56) is higher than v1 (6.93): the v2 build uses the coder imatrix (which optimizes for code and shifts wiki-text PPL) on top of the abliteration's distributional shift. v2's task benchmarks above are higher than v1's across HellaSwag, MMLU, and HumanEval, which is the divergence PPL cannot see.
Measured 2026-06-13 on a single RTX 3090 (24 GB), one llama-server, KV cache q8_0:
| metric | measured |
|---|---|
| decode speed | 36.5 tok/s |
| peak VRAM (4-slot serving) | 16.2 GB |
| max measured context (q8_0 KV) | 131,072 |
llama-server -m Qwen3.6-27B-Heretic-Cerebellum-v2-Q2_K_Mixed.gguf \
-ngl 99 --parallel 4 -c 24576 --jinja
This rig's measurements; no quality claims beyond them.
llama-server \
--model Qwen3.6-27B-Heretic-Cerebellum-v2-Q2_K_Mixed.gguf \
--mmproj Qwen3.6-27B-mmproj-BF16.gguf \
--n-gpu-layers 99 \
--ctx-size 8192 \
--jinja
--jinja is required for Qwen3.6. The enable_thinking chat-template flag
only takes effect when the Jinja template path is active; without it, the
model defaults to thinking mode on every request.
Non-thinking requests require an explicit flag at the API level:
{"chat_template_kwargs": {"enable_thinking": false}}
Qwen3.6 does not support the /think and /nothink soft-switch tokens
used by Qwen3.5. Thinking mode is on by default.
From the official Qwen3.6-27B documentation.
| Mode | temperature | top_p | top_k | min_p | presence_penalty | repetition_penalty |
|---|---|---|---|---|---|---|
| Thinking — general | 1.0 | 0.95 | 20 | 0.0 | 1.5 | 1.0 |
| Thinking — precise coding (WebDev) | 0.6 | 0.95 | 20 | 0.0 | 0.0 | 1.0 |
| Non-thinking (instruct) | 0.7 | 0.80 | 20 | 0.0 | 1.5 | 1.0 |
presence_penalty can be adjusted between 0 and 2 to reduce repetition loops;
higher values may occasionally cause language mixing.
Standard Cerebellum recipe. The tensor-type override file and ablation logs from the stock v4 build apply directly.
# 1. imatrix (constant ~300 MB RAM)
python -m osmosis.imatrix_stream \
--model Qwen3.6-27B-uncensored-heretic-v2-BF16.gguf \
--output imatrix.dat
# 2. quantize with stock llama-quantize
llama-quantize \
--imatrix imatrix.dat \
--tensor-type-file cerebellum_v5_qkvfix_overrides.txt \
Qwen3.6-27B-uncensored-heretic-v2-BF16.gguf \
Qwen3.6-27B-Heretic-Cerebellum-v2-Q2_K_Mixed.gguf \
Q2_K
The imatrix used for this build was generated from the coder corpus (ncall=8; same corpus as the stock Cerebellum v4 build).
The v5 (232-entry) and v4 (181-entry) tensor override files are included in this repository alongside the ablation logs.
Summary JSONs, per-question JSONL samples, EvalPlus eval JSON files, and
adversarial audit reports (AUDIT_*.md) are in benchmark_results/ in this
repository per project policy.