GLM-5.2-MaxMin108-GGUF

<!-- opening:start --> <p><a href="https://turintech.ai"><img src="https://huggingface.co/turintech/GLM-5.2-MaxMin108-GGUF/resolve/main/assets/turintech-logo.png" width="140" alt="TurinTech"></a><br><em>TurinTech models, built with Artemis</em></p>

Z.ai's GLM-5.2 on a single 128 GB machine, for chat, code and reasoning in English, expert-pruned by TurinTech from Unsloth's 2-bit GGUF.

• 95.78 GiB download for the default file, GLM-5.2-MaxMin108-IQ2_M-fast-00001-of-00006.gguf and 5 more parts.
• Fits a 128 GB machine: 104 of 121 GB on one NVIDIA GB10 (DGX Spark) at a 64K context.
• MMLU 61.84, against 62.75 for REAP-50 Q3_K_M (169.30 GiB), the closest public pruned file, which needs more than 128 GB.

<!-- opening:end -->

Run it

Needs llama.cpp with GLM-DSA support (build b10005 / 7f575c39d or newer) and a 128 GB machine. This command downloads the six default shards and starts the server on the first one.

hf download turintech/GLM-5.2-MaxMin108-GGUF \
  GLM-5.2-MaxMin108-IQ2_M-fast-00001-of-00006.gguf \
  GLM-5.2-MaxMin108-IQ2_M-fast-00002-of-00006.gguf \
  GLM-5.2-MaxMin108-IQ2_M-fast-00003-of-00006.gguf \
  GLM-5.2-MaxMin108-IQ2_M-fast-00004-of-00006.gguf \
  GLM-5.2-MaxMin108-IQ2_M-fast-00005-of-00006.gguf \
  GLM-5.2-MaxMin108-IQ2_M-fast-00006-of-00006.gguf \
  --local-dir . && \
  llama-server -m GLM-5.2-MaxMin108-IQ2_M-fast-00001-of-00006.gguf \
  --port 8080 --alias glm-5.2-maxmin108 -ngl 99 -fa 1 -ctk q8_0 -ctv q8_0 -c 65536 -np 1 -t 20 --jinja -rea off

<details> <summary>Settings, the reference file and Ollama</summary>

The repository also holds the reference file, GLM-5.2-MaxMin108-IQ2_M (six more shards, 100.33 GiB). Download one set, not both. The Hugging Face file sidebar adds the two sets together and shows them as one "IQ2_M" entry of about 211 GB.

Ollama: no verified supported package is provided here. Use the exact llama.cpp command above; an automatic integration button is not a compatibility test.

llama-server -m GLM-5.2-MaxMin108-IQ2_M-fast-00001-of-00006.gguf --port 8080 \
  --alias glm-5.2-maxmin108 -ngl 99 -fa 1 -ctk q8_0 -ctv q8_0 \
  -c 65536 -np 1 -t 20 --jinja -rea off

-rea off is recommended: without it, GLM-5.2's template emits a high reasoning-effort directive that can consume an entire short generation budget on hidden reasoning tokens.

Requires llama.cpp with GLM-DSA support (build b10005 / 7f575c39d or newer).

Questions, problems or a model to consider for a smaller version? Reply in our feedback thread.

</details> <!-- choose:start -->

Choose this model

• Best for running GLM-5.2 on a single 128 GB desktop such as one NVIDIA GB10 (DGX Spark).
• Consider another file when a 120B-class model fits your hardware: it will likely score higher. The 2-bit parent scores 87.29 on MMLU, and our IQ2_M reference file (100.33 GiB) is slightly more capable.

<!-- choose:end -->

Files

<!-- files:start -->

Every file we publish for this model. Sizes are download sizes, not memory.

• With llama.cpp or a Hugging Face pull, add the file's quant label as the tag: -hf turintech/GLM-5.2-MaxMin108-GGUF:IQ2_M-fast.

Without a tag, -hf and hf.co fetch GLM-5.2-MaxMin108-IQ2_M-00001-of-00006.gguf, not the default file, so always add the tag.

English only.

<!-- files:end --> <details> <summary>How the files compare</summary>

“Saved” is the file-size reduction versus the 2-bit parent.

IQ2_M-fast: Default. Smaller, frees about 5 GB. IQ2_M: The reference file used for the detailed measurements below.

Both keep the same 108 experts per layer, byte-identical; they differ only in the precision of the non-expert tensors. Each is a set of six shards: GLM-5.2-MaxMin108-IQ2_M-fast-0000N-of-00006.gguf for the default and GLM-5.2-MaxMin108-IQ2_M-0000N-of-00006.gguf for the reference. To get the reference instead:

hf download turintech/GLM-5.2-MaxMin108-GGUF --include "GLM-5.2-MaxMin108-IQ2_M-0000*" --local-dir .

The closest public alternative, REAP-50 Q3_K_M (169.30 GiB), scores 62.75 on MMLU against 63.00 for the reference file and is ahead on code (0.708), but does not fit a 128 GB machine.

</details> <!-- results:start -->

Results

Our default file against the closest public pruned file and Unsloth's 2-bit GGUF it was pruned from. There is no BF16 row: the 2-bit parent is the reference on this card.

Ours is the only one of the three that fits a 128 GB machine; expect much weaker world knowledge. How it was tested.

<!-- results:end --> <!-- chart:start -->

MMLU point estimates from the comparison table; uncertainty and evaluation settings remain below. The measured reference is Unsloth's 2-bit parent, not BF16. No BF16 or bartowski result is reported. Pruning trades accuracy for size. See How it was tested.

<!-- chart:end -->

<a id="benchmarks"></a>

<details> <summary>Evaluation details and limitations</summary>

How it was tested

Recommendation notes

Start here. Download the six GLM-5.2-MaxMin108-IQ2_M-fast shards (95.78 GiB in total) and start llama-server on shard 00001 with the command in Run it; llama.cpp finds the rest. It needs a 128 GB machine: on one NVIDIA GB10 (DGX Spark) it used 104 of 121 GB at a 64K context. It scores 61.84 on MMLU against 87.29 for the 2-bit parent, so expect much weaker world knowledge.

Use the exact IQ2_M-fast shard pattern below for the recommended build. Both builds share the IQ2_M label, so selecting that label alone does not identify a unique shard set.

Size and quality trade-offs

zai-org/GLM-5.2 is a 753B mixture-of-experts model. Unsloth's public 2-bit GGUF (UD-IQ2_M) is 222 GiB, so it needs a multi-GPU server. This file keeps 108 of the 256 experts in each layer and is 95.8 GiB, 57% smaller than that 2-bit file, so it runs on a single 128 GB desktop (one NVIDIA GB10 / DGX Spark). The cost is real: it scores 61.8 on MMLU against 87.3 for the parent, and 0.59 against 0.71 on MBPP. Use it when a task needs GLM-5.2 on one machine; if a 120B-class model fits the target hardware, that will likely score higher.

Everything below was run by us, in the same harnesses, for all four models, including the parent and the comparator. Nothing is quoted from another card.

MMLU: lm-evaluation-harness mmlu_generative, 5-shot, all 14,042 questions

MaxMin108 and REAP-50 differ by 0.25 MMLU points, with reported error bars of ±0.39. This does not establish equivalence. REAP-50 scores higher on STEM (+3.01) and humanities (+0.89). The table reports social sciences and other-category scores for both files. REAP-50 uses 3.82 bpw and 128 experts, compared with 2.65 bpw and 108 experts for MaxMin108.

MBPP: 3-shot, 500 problems, pass@1, code executed

REAP-50 is statistically indistinguishable from the parent on MBPP (0.708 vs 0.710) while losing 24.54 MMLU points. Pruning half the experts costs almost nothing on short-form code generation and roughly a quarter of world knowledge. That dissociation, where expert pruning destroys knowledge far faster than procedural coding ability, is the most transferable result here, and it holds for our artifact too: MaxMin108 retains 88.2% of parent MBPP against only 72.2% of parent MMLU.

MaxMin108 is genuinely behind REAP-50 on MBPP by 8.2 points, about 2.8 sigma. The MMLU comparison does not establish equivalence.

Generation quality: 13 frozen prompts, greedy decoding

Degeneration is scored objectively: 4-gram distinctness below 0.55, a line repeated 4 or more times, or a word repeated 5 or more times consecutively.

At greedy decoding, both artifacts here produced zero degenerate outputs across the thirteen tested prompts. This small sample does not establish reliability on other prompts.

</details>

Memory

Resident memory at a 64K context (-c 65536 -np 1 --jinja -rea off) on one NVIDIA GB10 (DGX Spark, 128 GB unified): GLM-5.2-MaxMin108-IQ2_M 109 of 121 GB, GLM-5.2-MaxMin108-IQ2_M-fast 104 of 121 GB. The parent and REAP-50 do not fit on this machine.

<details> <summary>File sizes and SHA-256 checksums</summary>

File checksums

SHA-256 and exact byte sizes for the published shards. Download all six shards of one build. The reference set totals 107,725,462,368 bytes (100.33 GiB); the fast set totals 102,844,865,312 bytes (95.78 GiB).

</details>

What is in the files

108 of the 256 routed experts per layer are kept, and they are byte-identical copies of Unsloth's UD-IQ2_M tensors; nothing in them is requantised. The -fast variant also stores the non-expert tensors at lower precision.

Known costs

• World knowledge is substantially reduced. On MMLU the default -fast file scores 61.84 and the reference file 63.00, against 87.29 for the parent. Answers can be fluent and confidently wrong. In our 13-prompt test neither file looped, which can make the remaining errors harder to spot.
• English only. GLM-5.2 is bilingual (en/zh); this release was built and tested in English only. Chinese was not evaluated.
• Code is behind REAP-50. MBPP pass@1 is 0.592 for the default -fast file and 0.626 for the reference file, against 0.708 for REAP-50.
• Derived from an IQ2_M quantisation, not BF16. All figures isolate the cost of pruning on top of quantisation; quantisation's own cost is not measured here.
• Two benchmarks. We attempted IFEval for instruction-following on two separate occasions and could not complete it: llama.cpp's server strictly parses model output against the template's declared reasoning and tool-call grammars, returning HTTP 500 on a parse failure. Roughly 1 to 4% of IFEval prompts triggered this on the pruned models across four server configurations, while the parent never did. That the compressed models occasionally emit unparseable structured output is itself a compression artifact, and one we would rather report than hide.
• Sampled decoding (--temp 0.6 --top-p 0.95) produces noticeably better output than greedy, so the greedy numbers above are a deliberately conservative floor.

Credits

This work rests on three others and would not exist without them:

• zai-org/GLM-5.2, the base model (MIT).
• unsloth/GLM-5.2-GGUF, the UD-IQ2_M quantisation these artifacts are sliced from. Every expert tensor here is a byte-identical copy of theirs, the released files still carry their importance-matrix metadata, and their dynamic-quantisation choices, meaning which tensors to protect at higher precision, are inherited wholesale.
• Cerebras REAP (ICLR 2026), the expert-pruning research this release builds on.
• pipenetwork/GLM-5.2-REAP50-Q3_K_M-GGUF, the comparator, and the reason a meaningful baseline existed to measure against. Its numbers here were measured by us; see that repository for its own licence and terms.

Custom versions

We also make fine-tuned and further optimised versions on request: tuned on specific data, built for the target hardware (CPU servers, laptops, Apple Silicon, edge devices), or compressed from a private model. Book a call with TurinTech.

About and licence

GLM-5.2 is Z.ai's general chat, code and reasoning model. This is an expert-pruned version of Unsloth's 2-bit GGUF of it: the experts kept are byte-identical copies, the rest are removed. English only.

This is an independent TurinTech release, not affiliated with or endorsed by Z.ai or Unsloth.

MIT, the same as zai-org/GLM-5.2; see LICENSE and NOTICE.

When serving these files to others, for example through an API, an inference provider or OpenRouter, we'd appreciate a mention of TurinTech (turintech.ai) as the source.

These files keep improving.