Downloads · 30 days
556
25% of all-time downloads
zen5-pro-gguf
zenlm/zen5-pro-gguf
zen5-pro-gguf is a text generation model from zenlm. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as mit.
High-quality Zen5 tier that fits on a single 128 GB unified-memory machine. Sparse MoE with 284B total / 37B active parameters per token, 1M context, asymmetric routed-MoE quantization (routed IQ2XXS up/gate, Q2K down…
All-time downloads
2.3K
Public
Repo size
604 GB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.gguf353 GB · 100%
At a glance
- Task
- Text Generation
- Library
- gguf
- License
- mit
- Access
- Public
- Created
- May 29, 2026
- Updated
- Jul 14, 2026
- SHA
- ffbf966a
Try a prompt
Base models
Model card
From the Hugging Face model README
Zen5 Pro
High-quality Zen5 tier that fits on a single 128 GB unified-memory machine. Sparse MoE with 284B total / 37B active parameters per token, 1M context, asymmetric routed-MoE quantization (routed IQ2_XXS up/gate, Q2_K down; shared experts, attention projections, routing logits and the LM head left at higher precision).
Repackaged from deepseek-ai/DeepSeek-V4-Flash (mit, DeepSeek) — quantized to GGUF from the abliterated variant by huihui-ai. Not trained from scratch — a permissively-licensed redistribution for the OSS-clean Zen model line.
Runs on a single 128 GB Apple Silicon (M3/M4 Max), DGX Spark (GB10), or H100 80 GB with the zen5-engine.
Part of the canonical Zen5 ladder:
| SKU | Hardware fit | This repo |
|---|---|---|
zen5-flash | anything | zen-5-flash-gguf |
zen5-mini | 32 GB | zen-5-mini-gguf |
zen5 (default) | 24 GB+ VRAM | zen-5-gguf |
zen5-pro | 128 GB single-machine | ← you are here |
zen5-max | 512 GB / 8x H100 | zen-5-max-gguf |
Files
| File pattern | Size | Notes |
|---|---|---|
*-IQ2XXS-w2Q2K-*-chat-v2-imatrix.gguf | 81 GB | Recommended — IQ2_XXS routed-MoE with imatrix calibration, fits 128 GB |
*-Layers37-42Q4KExperts-*-imatrix-fixed.gguf | 91 GB | Mixed-quant — higher quality at the boundary layers, fits 128 GB |
*-Q4KExperts-F16HC-*-imatrix.gguf | 153 GB | Q4-imatrix — for 256 GB+ unified-memory machines |
*-MTP-Q4K-Q8_0-F32.gguf | 3.5 GB | Optional speculative-decoding draft heads (use with --mtp) |
*-IQ2XXS-w2Q2K-*-chat-v2.gguf (non-imatrix) | 81 GB | Legacy Q2 — prefer the -imatrix version above |
*-Q4KExperts-F16HC-*-chat-v2.gguf (non-imatrix) | 153 GB | Legacy Q4 — prefer the -imatrix version above |
Run
Hosted via the Hanzo gateway (api.hanzo.ai) as zen5-pro.
Local with the zen5-engine:
git clone https://github.com/zenlm/zen5-engine
cd zen5-engine && make # macOS Metal
# or: make cuda-spark # DGX Spark GB10
# or: make cuda-generic # generic CUDA box
./download_model.sh q2-imatrix # pulls this repo's recommended GGUF
./zen5 -p "Hello"
./zen5-server --ctx 100000 --kv-disk-dir /tmp/zen5-kv --kv-disk-space-mb 8192
Performance (Metal, --ctx 32768 --nothink)
| Machine | Prompt | Prefill | Generation |
|---|---|---|---|
| MacBook Pro M3 Max 128 GB | short | 58.5 t/s | 26.7 t/s |
| MacBook Pro M3 Max 128 GB | 11709 tok | 250.1 t/s | 21.5 t/s |
| Mac Studio M3 Ultra 512 GB | short | 84.4 t/s | 36.9 t/s |
| Mac Studio M3 Ultra 512 GB | 12018 tok (Q4) | 448.8 t/s | 26.6 t/s |
| DGX Spark GB10 128 GB | 7047 tok | 343.8 t/s | 13.7 t/s |
License
apache-2.0 (this packaging). Upstream: deepseek-ai/DeepSeek-V4-Flash by DeepSeek, licensed MIT; abliterated variant by huihui-ai. This repository redistributes a quantized derivative; the upstream MIT terms are retained.
- Task
- Text Generation
- Library
- gguf
- License
- mit
- Languages
- en, zh
- Created
- May 29, 2026
- Updated
- Jul 14, 2026