Downloads · 30 days
100
35% of all-time downloads
Qwen3.8-27B-TokenRush-int4g128
zyhector/Qwen3.8-27B-TokenRush-int4g128
Qwen3.8-27B-TokenRush-int4g128 is a machine learning model from zyhector. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
4-bit weights for Token Rush, a single-stream inference engine for Qwen3.8-27B on one RTX 5090.
All-time downloads
284
Public
Repo size
17.1 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors17 GB · 100%
At a glance
- License
- apache-2.0
- Model type
- qwen3_5
- Access
- Public
- Created
- Sep 9, 2026
- Updated
- Sep 9, 2026
- SHA
- 29a49013
Base models
Model card
From the Hugging Face model README
Qwen3.8-27B-TokenRush-int4g128
4-bit weights for Token Rush, a single-stream inference engine for Qwen3.8-27B on one RTX 5090.
This checkpoint only loads in that engine. The packing is its own — group-wise
asymmetric int4 with a bf16 scale and a bf16 minimum per group of 128 along the
input dimension — so transformers, vLLM, SGLang and llama.cpp cannot read it.
| Format | int4, group 128, asymmetric: 4.25 bits per weight |
| Method | GPTQ with a per-row MSE range search; 256 x 2048 calibration tokens |
| Quantized | the 401 text-path matrices, lm_head included |
| Left bf16 | embeddings, norms, conv, the small GDN projections, the MTP head |
| Size | 17.0 GB |
| KL divergence to bf16 | 0.0232 mean over 81,920 positions |
| GSM8K | 96.5% against bf16's 96.0% (200 problems, greedy) |
Text path only; the vision tower is not included.
Details, the measurements behind those numbers, and the build recipe are in the repository.
Base model: Qwen/Qwen3.8-27B, Apache 2.0.
- Type
- qwen3_5
- License
- apache-2.0
- Created
- Sep 9, 2026
- Updated
- Sep 9, 2026