Downloads · 30 days
136
2% of all-time downloads
deucebucket/Qwen3-30B-A3B-Cerebellum-GGUF
Qwen3-30B-A3B-Cerebellum-GGUF is a machine learning model from deucebucket. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
<p align="center" <img src="cerebellumbanner.png" alt="Cerebellum" width="640" </p
Downloads · 30 days
136
2% of all-time downloads
All-time downloads
5.6K
Public
Repo size
35.6 GB
Likes
3
Public
Click a slice to open those files.
.gguf35.6 GB · 100%
From the Hugging Face model README
Ablation-guided mixed-precision quantization of Qwen/Qwen3-30B-A3B. 30B total parameters, 3B active (MoE with 128 experts, 8 active per token).
Instead of uniform quantization, we measure which weight groups survive aggressive compression and which don't. Groups that tolerate Q2_K get demoted; groups that don't stay at Q3_K_M or higher. The result: smaller files with less quality loss than uniform quants of the same size.
| File | Size | Description |
|---|---|---|
Qwen3-30B-A3B-Cerebellum-v3-Q3_K_M.gguf | 12 GB | Recommended — coder-optimized imatrix, improved allocation |
Qwen3-30B-A3B-Cerebellum-v2-Q3_K_M.gguf | 12 GB | Previous release |
Qwen3-30B-A3B-Cerebellum-v1-Q3_K_M.gguf | 9.6 GB | Maximum compression — all expert groups at Q2_K |
Evaluated using our standardized benchmark suite (ARC-Challenge, HellaSwag, MMLU-Redux, HumanEval+) with temperature=0, no thinking mode.
The model-index metadata in this card's frontmatter mirrors the recommended v3 build, measured with the local llama.cpp benchmark harness on RTX 3090. Full per-question artifacts are in benchmark_results/.
| Benchmark | Score | Questions |
|---|---|---|
| ARC-Challenge | 92.7% | 1,172 |
| HellaSwag | 83.8% | 10,042 |
| MMLU-Redux | 66.6% | 2,400 |
| HumanEval+ (base) | 75.0% | 164 |
| HumanEval+ (plus) | 70.7% | 164 |
| Model | Size | ARC | HellaSwag | MMLU | HumanEval |
|---|---|---|---|---|---|
| Q3_K_M (baseline) | 14 GB | — | — | — | — |
| Cerebellum v3 | 12 GB | 92.7% | 83.8% | 66.6%† | 75.0%‡ |
| Cerebellum v2 | 12 GB | 90.5% | 80.3% | 69.9%* | 72.0% |
| Cerebellum v1 | 9.6 GB | — | — | — | — |
* v2 MMLU = full MMLU (11,643 questions); v3 = MMLU-Redux (2,400 questions, harder subset)
‡ v3 HumanEval+ uses EvalPlus (stricter test cases than original HumanEval)
v3 improves +2.2 to +3.5 points over v2 on directly comparable benchmarks (ARC, HellaSwag).
v3 (coder imatrix) — Sacred (kept at Q3_K_M): attn_q, attn_v, ffn_down_exps
v3 — Demoted (Q2_K): attn_k, attn_output, ffn_gate_exps, ffn_up_exps
v2 (wiki imatrix) — Sacred (kept at Q3_K_M): attn_v, ffn_down_exps, output
v2 — Demoted (Q2_K): attn_k, attn_q, ffn_gate_exps, ffn_up_exps
All 48 layers treated uniformly (no per-layer variation needed for this model).
The key difference: v3 keeps attn_q at higher precision (swap with attn_output vs v2), driven by the coder-focused imatrix finding query projections more sensitive to quantization in code-focused inference.
Works with any llama.cpp-compatible tool:
# llama.cpp
./llama-server --model Qwen3-30B-A3B-Cerebellum-v3-Q3_K_M.gguf -ngl 99 --ctx-size 4096
# Ollama (create Modelfile pointing to the GGUF)
# LM Studio (drag and drop)
# koboldcpp, text-generation-webui, etc.
Quantized with Cerebellum — ablation-guided mixed-precision quantization by deucebucket.
Base model by Qwen.