Downloads · 30 days
31
27% of all-time downloads
Qwen3.8-27B-fp8
mahadev9/Qwen3.8-27B-fp8
Qwen3.8-27B-fp8 is a machine learning model from mahadev9. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
FP8 (W8A8 dynamic) quantization of Qwen/Qwen3.8-27B, produced with llm-compressor. lmhead, mlp.downproj, selfattn.oproj, and the first/last transformer layers are kept in the original precision - the layers most consi…
All-time downloads
114
Public
Parameters
26.9B
65.7 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors36.2 GB · 100%
Parameter types
How the weights are stored.
F8_E4M317.6B · 65%
Base models
Model card
From the Hugging Face model README
Qwen3.8-27B-fp8
FP8 (W8A8 dynamic) quantization of Qwen/Qwen3.8-27B,
produced with llm-compressor.
lm_head, mlp.down_proj, self_attn.o_proj, and the first/last
transformer layers are kept in the original precision - the layers most
consistently found to be quantization-sensitive - to recover
agentic/tool-calling quality.
Quality
KL divergence of the quantized model's next-token distribution vs the bf16
baseline (seq len 2048). prose is WikiText-2; agentic is tool-calling
turns (schema + query + call) rendered with the chat template.
| bucket | tokens | mean KL | top-1 agreement |
|---|---|---|---|
| prose | 24,130 | 0.0043 | 96.78% |
| agentic | 85,672 | 0.2864 | 90.55% |
| overall | 109,802 | 0.2244 | 91.92% |
Usage
vLLM:
vllm serve mahadev9/Qwen3.8-27B-fp8
transformers:
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained(
"mahadev9/Qwen3.8-27B-fp8", torch_dtype="auto", device_map="auto"
)
tok = AutoTokenizer.from_pretrained("mahadev9/Qwen3.8-27B-fp8")
- Type
- qwen3_5_text
- Created
- Sep 5, 2026
- Updated
- Sep 5, 2026