Downloads · 30 days
42
41% of all-time downloads
JasonW2025/Qwen3.6-35B-A3B-HybridQuant-gptq-dv41-KVcal
Qwen3.6-35B-A3B-HybridQuant-gptq-dv41-KVcal is a machine learning model from JasonW2025. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Identical weights to the production quant-qwen36-gptq-dv41 HybridQuant artifact (NVFP4 W4A16 MLP + FP8 attention, GPTQ, diverse-v4.1 calibration). The base artifact declares kvcachequantalgo: FP8 but ships no kscale/v…
Downloads · 30 days
42
41% of all-time downloads
All-time downloads
103
Public
Parameters
19.5B
23.4 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors23.4 GB · 100%
How the weights are stored.
U816.4B · 84%
From the Hugging Face model README
Identical weights to the production quant-qwen36-gptq-dv41 HybridQuant artifact
(NVFP4 W4A16 MLP + FP8 attention, GPTQ, diverse-v4.1 calibration). The base artifact
declares kv_cache_quant_algo: FP8 but ships no k_scale/v_scale, so vLLM serves
its fp8 KV cache at scale 1.0. This repo adds the missing scales, measured, not cast:
model-kvscales.safetensors: scale = amax x 1.25 / 448,
k_scale 0.0378–0.0558, v_scale 0.0156–0.0513.Using KV cache scaling factor 1.0 for fp8_e4m3 warning; this artifact does not,
with fp8 KV confirmed enabled in both. vLLM v0.25.1, GB10/sm_121a.Honest scope: measured amax is far below the e4m3 max (448), so scale 1.0 was not clipping this model. The scales recover the low tail of the distribution from subnormal flush. No accuracy delta has been benchmarked; do not cite one from this card.
Serve with --kv-cache-dtype fp8 (or leave auto; the checkpoint declares FP8 KV).