Downloads · 30 days
15
4% of all-time downloads
d-alistarh/Qwen3-8B-GSQ-Q3_K_M-GGUF
Qwen3-8B-GSQ-Q3_K_M-GGUF is a machine learning model from d-alistarh. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Quantized with GSQ (Gumbel-Softmax Quantization) using the Q3KM mixed-precision recipe.
Downloads · 30 days
15
4% of all-time downloads
All-time downloads
382
Public
Repo size
5.1 GB
Likes
0
Public
Click a slice to open those files.
.gguf5.1 GB · 100%
From the Hugging Face model README
Quantized with GSQ (Gumbel-Softmax Quantization) using the Q3_K_M mixed-precision recipe.
| Model | Size | PPL | Delta vs F16 |
|---|---|---|---|
| F16 baseline | 16 GB | 10.36 | — |
| GSQ (this model) | 4.74 GB | 10.59 | +0.23 (+2.2%) |
| Unsloth Dynamic Q3_K_M | ~4 GB | 10.90 | +0.54 (+5.2%) |
| Standard Q3_K_M | ~4 GB | 11.39 | +1.03 (+9.9%) |
Per-projection assignment (uniform across all 36 layers):
Effective average: ~3.4 bits per weight.
GSQ applies GPTQ initialization followed by 2000 steps of Gumbel-Softmax refinement per linear layer, optimizing quantized weights against the Hessian-weighted reconstruction loss.