Downloads · 30 days
0
megasoft78/gemma-4-26B-A4B-it-attnQ4K-GGUF
gemma-4-26B-A4B-it-attnQ4K-GGUF is a machine learning model from megasoft78. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as gemma.
A requantized variant of unsloth/gemma-4-26B-A4B-it-GGUF's UD-IQ2M build, built for local coding-agent use on Apple Silicon unified memory (bandwidth-bound decode).
Downloads · 30 days
0
Access
Public
Updated Sep 10, 2026
Repo size
—
Likes
0
Public
Click a slice to open those files.
.md3.2 KB · 67%
From the Hugging Face model README
A requantized variant of unsloth/gemma-4-26B-A4B-it-GGUF's
UD-IQ2_M build, built for local coding-agent use on Apple Silicon unified memory (bandwidth-bound decode).
Only the always-on tensors — attention (attn_q/k/v/output) and the tied token_embd/lm_head — are
requantized from Q5_K down to Q4_K. Every routed expert tensor is left byte-for-byte untouched. This
model is a Mixture-of-Experts architecture where only a small fraction of experts activate per token, but the
attention and embedding tensors are read on every token regardless of which experts route. On a
bandwidth-bound decode workload (Apple unified memory, no dedicated VRAM), cutting bytes in the always-on
path converts directly into decode speed, without touching the expert precision that carries most of this
model's task quality.
gemma-4-26B-A4B-it-UD-IQ2_M-attnQ4K.gguf, 9,777,197,536 bytes (~9.1 GiB), down from the base
UD-IQ2_M's 10,014,755,296 bytes (~9.3 GiB).llama-quantize --allow-requantize --tensor-type-file, using unsloth's own published imatrix
for calibration. The exact tensor-type map is published alongside the tool that produces this file:
optional/attn-embed-q4k.tensor-types.txt
in gemma4-coding-kit — fully reproducible from the
base model plus that recipe file.Benchmarked on a Mac mini M1 (16GB unified memory, 68GB/s bandwidth) against gemma4-coding-kit's own 9-scenario / 44-bug multi-file coding-agent benchmark suite (symptom-mode, single-shot, temperature 0):
| Config | Score (counted) | Decode speed |
|---|---|---|
Base UD-IQ2_M (unmodified) | 36/38 | 26.6 tok/s |
This requant (attnQ4K) | 36/38 | ~26.3–28.1 tok/s (3 independent runs) |
Identical benchmark score to the unmodified base model — quality on this suite is unaffected — at the same or better decode speed. This is the model gemma4-coding-kit ships by default; see that repo for the full benchmark methodology, per-bug results, and setup instructions.
llama-server -m gemma-4-26B-A4B-it-UD-IQ2_M-attnQ4K.gguf -ngl 99 -fa on -c 24576 -ub 256 -b 256 \
--no-warmup -np 1 --spec-type ngram-simple --reasoning off --ctx-checkpoints 0 --cache-ram 0
Or just run gemma4-coding-kit's setup.sh, which
downloads and configures this file automatically.
Base quantization and imatrix: unsloth (unsloth/gemma-4-26B-A4B-it-GGUF).
Base model: Google's Gemma 4 (subject to the Gemma license). This repo
only modifies the attention/embedding tensor precision on top of unsloth's existing quantization and imatrix.