GLM-5.2-Demolition-q4a4-soul-MLX

GLM-5.2-Demolition · q4a4-soul (v3)

Explore the model guide · All public work

Release at a glance

A demolition of zai-org/GLM-5.2 (744B total / 39B active MoE, MIT) down to a ~105 GB (98 GiB) 4-bit model that loads and runs fully on a single Apple M5 Max (128 GB). v3's distinguishing move is soul-targeted expert pruning — the kept experts are chosen by saliency measured on our facet data, not a generic corpus — plus a deliberately pure vanilla-code core with swappable heritage "souls" mounted on demand.

Download scope

The root model tensors total 104.89 GB (97.68 GiB). This repository also contains nested adapter checkpoints: all .safetensors files together total 187.51 GB. An unfiltered download retrieves those additional experiments too. This does not change the core model's memory use. Select core files first, then download a specific adapter if your recipe needs it:

hf download philipjohnbasile/GLM-5.2-Demolition-q4a4-soul-MLX \
  --include 'model*.safetensors' --include 'model.safetensors.index.json' \
  --include 'config.json' --include 'tokenizer*' --include 'chat_template.jinja' --include '*.py' --include '*.md' \
  --local-dir ./glm52-demolition-core

This is a download command, not a validated launch command. Read the bundled GLM DSA setup and extended model card before running it.

Architecture: a PURE core + swappable souls

• CORE (always-on): just the vanilla languages, done excellently — Python · TypeScript · JavaScript · Rust · Go · HTML · CSS · SQL · Postgres. No frameworks, no baked specialties. Latest versions, vanilla stdlib.
• SOULS (mount per-request — the model factory): small LoRA adapters that name a field's masters to activate latent eliteness:
  • art (Basquiat/Haring/Banksy/Sol LeWitt/Casey Reas) · music (Bach/J Dilla/Eno/WALL-E) · design (Rams/Bauhaus) · perfumery (Beaux/Guerlain/Ellena) · science (Feynman/Darwin/Sagan) · legacy (K&R/Knuth/Dijkstra/Hopper) · security (Saltzer-Schroeder/Aleph-One, purple-team) · gamedev (Carmack/Handmade-Hero, vanilla from-scratch) · fullstack (htmx/Go-stdlib/Postgres) · math · dataviz · prose · architecture · research.

Running the full 744B instead

This repo is a pruned ~105 GB (98 GiB) MLX build; its weights are not compatible with iliria.

If you want the full 744B GLM-5.2 (unpruned, int4) on the same 128 GB Mac, iliria streams the full model's experts from SSD. A ready-to-run container is at GLM-5.2-colibri-int4-with-int8-mtp — ~384 GB, no conversion needed.

The demolition lineage (honest)

Why 4-bit, not 3: 3-bit was just below the quality cliff; 4-bit is just above it and MLX's best-optimized kernel (cleanest packing). 2-bit is worse. These experiments did not establish superiority over an appropriately sized unpruned model; this artifact is one measured pruning-and-quantization experiment; it does not establish an optimal recipe.

Method

1. Saliency (23_stream_calibrate) on our facet corpus → score each routed expert.
2. Prune (24_apply_prune --ratio 0.77) → keep the top-saliency experts.
3. Re-quantize (24b_stream_requantize --bits 4) → uniform 4-bit experts, 4-bit attn, 6-bit head.
4. Heal (06_heal_lora) — LoRA on vanilla FOCUS-9 gold; souls heal separately per facet.

Honest scope

• Speed: ~11–14 tok/s single-stream — memory-bandwidth-bound (recorded on M5 Max; an earlier version of this line said ~10; the recorded speculative-decoding gain was only ~1.05× here, so it's not used).
• Strengths: the FOCUS-9 vanilla languages + whichever soul is mounted. Not general trivia — those experts were deliberately pruned. Best driven by a verifier-first agent (the compiler steers each line).
• Eval: HumanEval-164 pass@1 = 114/164 (69.5%) — full set, single-shot, scored on hidden tests by real verifiers (the easy n=20 subset was 95%). This is a result on the stated single-shot protocol, not a controlled comparison with GPT-4 or current frontier models. An agent loop was not quantified by this single-shot score. Strong on writing vanilla FOCUS-9 functions from a spec; weaker on hard debugging/multi-step and off-distribution prompts. Honest scope: this is a pruned research artifact. The published extended model card contains additional project context; MISSION_SUMMARY.md is not included in this Hub revision.

Built with the open pipeline at glm52-demolition. Public (MIT — GLM-5.2 is Z.ai Pure-Open).