Downloads · 30 days
9.5K
15% of all-time downloads
jrell/Qwen3.8-27B-i1-IQ4_XS-GGUF-Smaller
Qwen3.8-27B-i1-IQ4_XS-GGUF-Smaller is a machine learning model from jrell. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
A custom hybrid quantization of the Qwen3.8-27B base model, specifically designed to fit Multi-Token Prediction (MTP) and long context into a strict 16GB VRAM hardware budget (like consumer RTX 4080 / RTX 5080 cards).
Downloads · 30 days
9.5K
15% of all-time downloads
All-time downloads
62.1K
Public
Repo size
13.5 GB
Likes
97
Public
Click a slice to open those files.
.gguf13.5 GB · 100%
From the Hugging Face model README
A custom hybrid quantization of the Qwen3.8-27B base model, specifically designed to fit Multi-Token Prediction (MTP) and long context into a strict 16GB VRAM hardware budget (like consumer RTX 4080 / RTX 5080 cards).
Instead of compressing the whole model equally, it is split by layer type:
IQ4_XS to protect reasoning, coding, and formatting logic.IQ3_S to trim file size. The trade-off is a minor loss in general knowledge and long-context recall, so if you use the model for something like creative writing, you'd probably be better off just using normal IQ4_XS quants.The model was compiled using mainline llama.cpp.
/home/llama.cpp/build/bin/llama-quantize \
--imatrix Qwen3.8-27B.imatrix.mradermacher-ubergarm.dat \
--tensor-type ffn_down=iq3_s \
--tensor-type ffn_up=iq3_s \
--tensor-type ffn_gate=iq3_s \
BF16/Qwen3.8-27B-BF16.gguf \
Qwen3.8-27B-i1-IQ4_XS-GGUF-Smaller.gguf \
IQ4_XS \
64