GLM-5.2-REAP50-Q2_K-GGUF

GLM-5.2-REAP50-Q2_K-GGUF

GGUF of GLM-5.2, REAP expert-pruned (50%) + Q2_K (~129 GB) — the maximum-context option for 2× 96 GB GPUs (192 GB), leaving ~60 GB for KV cache (≈2.5× the context room of the Q3_K build).

⚠️ Quality: fragile — most-degraded variant

This stacks REAP-50 (~+37.5% perplexity vs full GLM-5.2) with 2-bit Q2_K. It works but is delicate:

• ✅ Coherent with sampling: "The capital city of France … is Paris."
• ❌ Collapses into repetition (* * * *) with greedy / temp 0 decoding.

Use these sampler settings (or it may loop):

--temp 0.6 --repeat-penalty 1.1 --top-p 0.95

If you can spare the VRAM, the Q3_K_M build (~169 GB) is noticeably more robust. For real quality, use the MLX REAP-25 (+2.3% PPL) or full GLM-5.2.

Requires a patched llama.cpp

Stock llama.cpp can't load GLM-5.2 GGUFs yet (DSA indexer required on every layer; GLM-5.2 ships it on only some). Apply the included llama.cpp-glm-dsa-indexer-optional.patch (or wait for ggml-org/llama.cpp#24770), rebuild, then:

./build/bin/llama-cli -m GLM-5.2-REAP50-Q2_K-00001-of-00004.gguf --jinja -ngl 99 \
  --temp 0.6 --repeat-penalty 1.1 -p "..."

REAP-50 = top-128 of 256 experts/layer by saliency; runs as full MLA attention. Smoke-tested on Metal (~20 tok/s).