gemma-4-21b-a4b-it-REAP-GGUF

Gemma 4 21B-A4B-it REAP — GGUF Quantizations

GGUF quantizations of 0xSero/gemma-4-21b-a4b-it-REAP — a 20% expert-pruned Gemma 4 using Cerebras REAP.

If you have 16GB VRAM or less, consider the 19B variant instead — it's dramatically faster due to VRAM headroom.

Available Quantizations

Performance (AMD RX 9070 XT, 16GB VRAM, Vulkan)

Note: On 16GB cards the 21B Q4_K_M still hits 99% VRAM, limiting the speedup. The 19B variant drops to 76% VRAM and runs 7x faster. The 21B shines on 24GB+ cards where VRAM isn't the bottleneck.

Quick Start

# Download
hf download vsark/gemma-4-21b-a4b-it-REAP-GGUF gemma-4-21b-reap-Q4_K_M.gguf

# Run (IMPORTANT: use --reasoning off for Gemma 4)
llama-server \
    --model gemma-4-21b-reap-Q4_K_M.gguf \
    --n-gpu-layers 99 \
    --ctx-size 8192 \
    --reasoning off \
    --host 127.0.0.1 --port 8012

Important Notes

• Use --reasoning off with llama-server
• Requires recent llama.cpp with gemma4 architecture support
• 20% pruning preserves more knowledge than the 30% variant — better for trivia/knowledge-intensive tasks

Verification

SHA256 checksums:

926f03ac6a1995d72e05b1dc08025dae47a447469b5ccf9840bcce2846c54e55  gemma-4-21b-reap-Q4_K_M.gguf
30ee2997d7ecd3fd8ef5ad04ce9a1890f3ab2536ae8da9ab50985a810456fd63  gemma-4-21b-reap-Q3_K_S.gguf

Credits

• REAP pruning: 0xSero
• Base model: Google Gemma 4
• GGUF conversion + benchmarks: vsark