Downloads · 30 days
83
5% of all-time downloads
deucebucket/Gemma-4-E4B-it-Heretic-Cerebellum-GGUF
Gemma-4-E4B-it-Heretic-Cerebellum-GGUF is a text generation model from deucebucket. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as apache-2.0.
<p align="center" <img src="cerebellumbanner.png" alt="Cerebellum" width="640" </p
Downloads · 30 days
83
5% of all-time downloads
All-time downloads
1.5K
Public
Repo size
4.5 GB
Likes
4
Public
Click a slice to open those files.
.gguf4.5 GB · 100%
From the Hugging Face model README
Sensitivity-guided mixed-precision quantization of coder3101/gemma-4-E4B-it-heretic, which is itself a decensored variant of google/gemma-4-E4B-it produced by coder3101 using Heretic v1.2.0.
All future Heretic versions of this build will live in this repository. Version identifiers appear only in filenames, not in the repo name.
| File | Size | Description |
|---|---|---|
Gemma-4-E4B-it-Heretic-Cerebellum-v1-Q3_K_M.gguf | 4.2 GiB (~4.51 GB, 4,498,725,440 bytes) | Cerebellum v2 recipe — recommended |
No vision mmproj is included. Stock Cerebellum v2 ships none either — Gemma 4 E4B's vision projector is not publicly available in a distributable form.
Benchmarks run on these GGUF files directly using llama.cpp on RTX 3090.
All numbers are audited; every failed answer was manually verified as a genuine
model error — audit reports are in benchmark_results/AUDIT_e4b.md.
Full per-question detail (summary JSON, samples JSONL, EvalPlus eval JSON,
adversarial audit reports) is in benchmark_results/ in this repository.
| Benchmark | Heretic Cerebellum v1 (4.2 GiB) | Stock Cerebellum v2 (4.2 GiB) | Notes |
|---|---|---|---|
| Wiki PPL (ctx 2048, 32 chunks) | 50.61 ± 1.23 | 51.35 ± 1.26 | RTX 3090, identical invocation — see note |
| ARC-Challenge | 87.37% (1172 q) | 85.7% | 25-shot |
| HellaSwag | 74.98% (10042 q) | 75.3% | 10-shot |
| MMLU-Redux | 58.63% (2400 q) | 58.4% | 5-shot — see footnote |
| HumanEval base (chat, no-think) | 70.12% (164 q) | 68.3% | pass@1, patched evalplus harness |
| HumanEval+ (chat, no-think) | 65.24% | not published | pass@1, patched evalplus harness |
Stock Cerebellum v2 is the same tensor allocation applied to the non-heretic base.
Stock v2's published card number of 52.20 was measured under a different protocol. The 51.35 figure above is a same-night, same-invocation pair run alongside the heretic build (ctx 2048, 32 chunks, RTX 3090) and is the directly comparable baseline. The heretic build's 50.61 ± 1.23 is 0.74 points lower than that paired measurement; error bars overlap at the edges, making this a marginal improvement consistent with the abliteration applying a slight distributional shift rather than introducing noise.
The audit (BE-16) identified 12 truly identical duplicate entries in the MMLU-Redux dataset resulting from a subject-boundary edge case in the benchmark runner. These entries inflate the reported score by ≤0.04pp (58.6654% → 58.6250% without duplicates), which is below the 0.1pp non-blocking threshold. The reported 58.63% is rounded from 58.625% and is trustworthy within standard rounding tolerance.
Zero fabricated completions. One model-authored stub (HumanEval/79,
decimal_to_binary) — a literal pass placeholder — was correctly scored as
fail and does not inflate pass@1. Stock v2's published 68.3% base was measured
with the upstream evalplus harness; HumanEval+ was not published for stock v2.
The heretic numbers above use the patched no-think chat harness (same harness
used for all Cerebellum builds since the harness audit, May 2026).
The following parameters are as reported in coder3101's model card and are reproduced here for downstream reference.
| Parameter | Value |
|---|---|
| Method | Arbitrary-Rank Ablation (ARA) with row-norm preservation |
| Targeted layers | 21–42 |
| preserve_good_behavior_weight | 0.3873 |
| steer_bad_behavior_weight | 0.0003 |
| overcorrect_relative_weight | 0.8555 |
| neighbor_count | 14 |
Tool: Heretic v1.2.0, method: Arbitrary-Rank Ablation (ARA) with row-norm preservation.
Same allocation as the stock build. Listed here for reference. 179 overrides over a Q3_K_M base: 174 PLE tensors pinned to Q5_K, 2 sacred promotions to Q6_K, 3 demotions to Q2_K.
| Group | Precision | Rationale |
|---|---|---|
| Per-layer embedding (PLE) tensors × 174 | Q5_K | Without PLE protection: PPL ~104 at Q3_K_M; with PLE@Q5_K: PPL ~55. Q4_K→Q3_K cliff is catastrophic for this architecture |
| 2 sacred attention tensors | Q6_K | Highest per-tensor PPL sensitivity per ablation |
| 3 ffn tensors | Q2_K | Reverse ablation confirmed Q2_K neutral or beneficial at these positions |
| Norm tensors | F32 | Protected; standard practice |
| Bulk ffn / remaining attention | Q3_K_M | Base precision, imatrix-guided |
The PLE protection is the load-bearing element of this recipe. Gemma 4's per-layer embedding tensors degrade catastrophically at Q3_K_M — the Q4_K→Q3_K transition produces a PPL cliff rather than a gradual slope, going from ~55 to ~104 without PLE pins. This finding is documented in the stock v2 ablation logs.
Measured 2026-06-13 on a single RTX 3090 (24 GB), one llama-server, KV cache q8_0:
| metric | measured |
|---|---|
| decode speed | 116 tok/s |
| peak VRAM (4-slot serving) | 5.3 GB |
| max measured context (q8_0 KV) | 65,536 |
llama-server -m Gemma-4-E4B-it-Heretic-Cerebellum-v1-Q3_K_M.gguf \
-ngl 99 --parallel 4 -c 24576 --jinja --reasoning-budget 0
This rig's measurements; no quality claims beyond them.
llama-server \
--model Gemma-4-E4B-it-Heretic-Cerebellum-v1-Q3_K_M.gguf \
--n-gpu-layers 99 \
--ctx-size 8192
Fits comfortably in 6 GB VRAM at 8K context on an RTX 3060/3070/3080 class card.
Standard Cerebellum recipe. The tensor-type override file and ablation logs from the stock v2 build apply directly.
# 1. imatrix (constant ~300 MB RAM)
python -m osmosis.imatrix_stream \
--model gemma-4-E4B-it-heretic-f16.gguf \
--output imatrix.dat
# 2. quantize with stock llama-quantize
llama-quantize \
--imatrix imatrix.dat \
--tensor-type-file cerebellum_v2_overrides.txt \
gemma-4-E4B-it-heretic-f16.gguf \
Gemma-4-E4B-it-Heretic-Cerebellum-v1-Q3_K_M.gguf \
Q3_K_M
The 179-entry tensor override file (cerebellum_v2_overrides.txt) is included
in this repository alongside the ablation logs.
Summary JSONs, per-question JSONL samples, EvalPlus eval JSON files, and
adversarial audit reports (AUDIT_e4b.md) are in benchmark_results/ in this
repository per project policy.