GLM-5.2-colibri-int4-with-int8-mtp

ℹ️ This is an unmodified mirror

I did not create these weights. This repo is a byte-identical re-host of mateogrgic/GLM-5.2-colibri-int4-with-int8-mtp, published only as an extra copy for availability. All 145 files match the source SHA256 hashes exactly — nothing was requantized, repacked, or altered.

Please prefer the original repo — credit and download counts belong there.

Lineage: zai-org/GLM-5.2-FP8 → jlnsrk/GLM-5.2-colibri-int4 (colibrì int4 conversion) → mateogrgic/…-with-int8-mtp (int8 MTP heads) → this mirror.

Identical copies also exist at AIA-YSC, gubernac, and others.

Everything below is the original author's model card, preserved as-is.

---

GLM-5.2 — colibrì int4 container (~370 GB)

This is the *EXACT SAME THING* as https://huggingface.co/jlnsrk/GLM-5.2-colibri-int4, BUT with int8 MTP heads, which are needed for speculative decoding—and with that, an overall major inference speedboost. The original int4 MTP heads have low acceptance rate, and are essentially useless.

⚠️ This is NOT a GGUF / AWQ / GPTQ / MLX model. It only works with the colibrì engine.

Usage

# Get the engine
git clone https://github.com/JustVugg/colibri && cd colibri/c && ./setup.sh

# download this repo to a FAST local disk
hf download mateogrgic/GLM-5.2-colibri-int4-with-int8-mtp --local-dir /nvme/glm52

# run on CPU
COLI_MODEL=/nvme/glm52 ./coli chat

Requirements: Linux (or WSL2), gcc + OpenMP, AVX2, ≥16 GB RAM, ~400 GB free NVMe.

Provenance & license

Converted from zai-org/GLM-5.2-FP8 (MIT). This derivative is likewise MIT. Conversion performed with colibrì's official converter, unmodified. Cloned & modded from jlnsrk/GLM-5.2-colibri-int4