Downloads · 30 days
5.9K
15% of all-time downloads
GLM-5.2-colibri-int4
jlnsrk/GLM-5.2-colibri-int4
GLM-5.2-colibri-int4 is a machine learning model from jlnsrk. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
Pre-converted weights for colibrì — the pure-C engine that runs GLM-5.2 (744B MoE) on a consumer machine with ~25 GB of RAM by streaming routed experts from disk.
All-time downloads
38.7K
Public
Repo size
381 GB
Likes
171
Trending 1
Repo makeup
Click a slice to open those files.
.safetensors379 GB · 100%
At a glance
- License
- mit
- Model type
- glm_moe_dsa
- Access
- Public
- Created
- Jul 6, 2026
- Updated
- Jul 18, 2026
- SHA
- cce7e760
Base models
Model card
From the Hugging Face model README
GLM-5.2 — colibrì int4 container (~370 GB)
Pre-converted weights for colibrì — the pure-C engine that runs GLM-5.2 (744B MoE) on a consumer machine with ~25 GB of RAM by streaming routed experts from disk.
This is the output of colibrì's coli convert (convert_fp8_to_int4.py --ebits 4 --io-bits 8, including the MTP head for native speculative decoding), uploaded so you don't have to download the 756 GB FP8 checkpoint and spend a day converting it.
⚠️ This is NOT a GGUF / AWQ / GPTQ / MLX model. It is colibrì's own container: for each quantized weight, name (U8, packed int4 nibbles) + name.qs (F32 per-row scales), quantized with math bit-identical to the engine's C kernels. It only works with the colibrì engine.
Usage
# get the engine
git clone https://github.com/JustVugg/colibri && cd colibri/c && ./setup.sh
# download this repo to a FAST local disk (NVMe, ext4 — never a network/9p mount)
hf download jlnsrk/GLM-5.2-colibri-int4 --local-dir /nvme/glm52_i4
# chat (RAM budget, expert cache and MTP auto-detected)
COLI_MODEL=/nvme/glm52_i4 ./coli chat
Requirements: Linux (or WSL2), gcc + OpenMP, AVX2, ≥16 GB RAM, ~400 GB free NVMe.
What's inside
| file | contents |
|---|---|
out-*.safetensors | dense weights (attention/MLA, shared experts, embeddings) + 21,504 routed experts, int4 per-row scales; router/norms kept F32 |
| MTP shard | GLM-5.2's multi-token-prediction head (layer 78) — enables lossless speculative decoding (~2 tok/forward) |
config.json, tokenizer*.json, generation_config.json | copied from the base repo |
Conversion: FP8 (e4m3, 128×128 block scales) → f32 → int4 with np.rint matching the engine's lrintf — token-identical to converting locally.
Provenance & license
Converted from zai-org/GLM-5.2-FP8 (MIT). This derivative is likewise MIT. Conversion performed with colibrì's official converter, unmodified.
- Type
- glm_moe_dsa
- License
- mit
- Languages
- en, zh
- Created
- Jul 6, 2026
- Updated
- Jul 18, 2026