GLM-5.1-REAP-Solidity-MXFP8

GLM-5.1-REAP-Solidity-MXFP8

A proof-of-concept that turns GLM-5.1 into a smaller, domain-focused model for on-device Solidity and smart-contract engineering on Apple Silicon. Instead of relying only on lower-bit quantization, this checkpoint uses targeted MoE expert pruning: keep the experts most active for the target workload, remove the least relevant routed experts, and retain higher precision for the remaining model.

Built with mlx_fun - an MLX-native toolkit for MoE quantization, expert pruning (REAP/REAM), and online expert-activation collection - by sombrax.com.

<div align="center"> <img src="./assets/expert_activation_heatmap.png" alt="Expert activation heatmap (75 MoE layers × 256 experts, weighted-frequency rank-sum)" width="900"/> </div>

Each row is one of the 75 MoE layers; each column is one of the 256 routed experts in the base model. Green indicates higher observed saliency for the calibration workload; red indicates lower observed saliency. The released checkpoint keeps 128 of 256 routed experts per MoE layer.

---

Why this exists

The privacy push is real: people increasingly want to run capable models on their own laptop, not ship every prompt to a remote API. The wall most people hit is GPU memory. Today's answer is to quantize aggressively — Q3, even Q2 — and accept across-the-board precision loss. The model gets dumber at everything, including the things you actually wanted it to do.

There's a different lever. A Mixture-of-Experts model is, by construction, a collection of specialists. Most prompts only fire a small subset of experts. For a focused workload - coding in one language, auditing one contract type, writing in one style - the bulk of the model is dead weight specific to your use case. Trim those experts, keep the ones that fire, and you get back the fidelity you spent on bits.

That's what this model is. Same precision (8-bit MXFP8), half the experts, biased toward experts observed during Solidity-focused calibration.

What this model is

The expert count per layer is uniform (128 everywhere), but which 128 experts are kept differs per layer - each layer keeps its own top-128 by saliency. The router and the score-correction bias are sliced to match.

Build summary

This checkpoint was calibrated on a focused mix of Solidity-oriented coding, contract-analysis, and test-generation workloads. Routed-expert saliency was collected from representative model traffic and Solidity source samples, then aggregated per MoE layer. The least-salient 128 routed experts were removed from each MoE layer while preserving the dense layers, router alignment, and top-k routing behavior.

Private prompts, internal task traces, target contracts, and operational pipeline details are intentionally omitted from this public model card. The general method can be reproduced with mlx_fun: collect routed-expert activation statistics for your own domain corpus, quantize the base model to the target format, then apply per-layer REAP pruning with the collected saliency.

Use cases this is for

• On-device Solidity audit assistants. Smart-contract review, vulnerability triage, fork-test scaffolding.
• Local coding agents in general where you'd otherwise reach for a smaller / dumber model. The methodology generalizes - swap calibration data and re-prune.
• Research into routing-based pruning of MoEs on consumer-class hardware.

What it isn't

• Not a fine-tune. Weights aren't moved; experts that don't help your workload are simply removed.
• Not a general-purpose chatbot anymore. Topics far outside the calibration mix (poetry, legal, biology trivia) will degrade more than full GLM-5.1 does. That's the point.
• Not a replacement for evaluation - REAP at 50% expert reduction is aggressive; benchmark on your actual workload before deploying.
• Not a security-certified auditor. It can miss vulnerabilities, invent false positives, or generate unsafe test assumptions. Treat outputs as review assistance only; smart-contract findings still need human verification.

Evaluation snapshot

The following results come from two end-to-end Solidity-agent tasks on a representative public DeFi exploit case. The same task definitions were run against this local checkpoint, a same-base local Q3 checkpoint without expert pruning, z.ai direct hosted GLM-5.1, and one hosted FP8 reference configuration.

These numbers are included to give a practical workload signal. They are not a formal benchmark suite, not a provider benchmark, and not a security certification. Hardware for the local runs was an Apple M3 Ultra system with 512 GB unified memory. Hosted-reference results should be treated as point-in-time observations; the audit-task hosted runs were performed during peak service hours.

Contract investigation task

The agent reviewed a complex contract execution path and produced a structured finding report.

The pruned checkpoint was the only local run that completed the required structured output. Compared with the local Q3 baseline, it used ~17% fewer total tokens and ~27% fewer tool calls.

The z.ai direct hosted run failed the audit task in both attempts during peak service hours, despite using the hosted full GLM-5.1 service. That failure is notable because the same z.ai configuration completed the Solidity fork-test task successfully below.

Fork-test generation task

The agent generated a Foundry fork test, iterated through build/runtime errors, and produced a passing reproduction test.

All configurations reached the same expected outcome on this task. Compared with the local Q3 baseline, this model was ~60% faster wall-clock, used ~69% fewer total tokens, and made ~55% fewer tool calls.

This is not a head-to-head against full-precision GLM-5.1, because the unpruned MXFP8 checkpoint exceeds the available memory of the local test machine. The snapshot shows that REAP pruning preserved the ability to complete these representative Solidity-agent tasks while reducing local token and tool-call usage versus the same-base Q3 baseline.

Limitations and notes

• Calibration used a lower-bit capture checkpoint, then the resulting saliency was applied to the MXFP8 target. Routing saliency is generally stable across quantization levels, but stricter reproductions should collect saliency directly on their target quantization.
• Per-layer expert count is uniform but the kept indices differ per layer. Tooling that assumes "expert N is the same thing everywhere" will not hold here.
• Smoke-tested locally with mlx-fun smoke-test; produces fluent, syntactically valid Solidity from a pragma solidity ^0.8.0; prompt. Quantitative benchmarks against unpruned GLM-5.1 MXFP8 weren't run on the build machine because the unpruned checkpoint exceeds available unified memory.
• The benchmark above is a single real case, not a corpus. Before using this model for production audit work, evaluate it on your own vulnerability set, fork-test tasks, and false-positive tolerance.

License

MIT, inheriting from the base model. Original GLM-5.1 license terms apply to the base weights.

Links

• Toolkit: github.com/dexloom/mlx_fun
• Base model: zai-org/GLM-5.1
• Made by: sombrax.com
• Issues / discussion: open one on the mlx_fun repo

If this model or workflow is useful, please star the mlx_fun repo.