Downloads · 30 days
16
29% of all-time downloads
kwanhee/Qwen3-30B-A3B-P48NVFP4-W4A4-GS32-GW2
Qwen3-30B-A3B-P48NVFP4-W4A4-GS32-GW2 is a machine learning model from kwanhee. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Refinement with a gate-weighted per-expert loss (gateweightexponent = 2): each (token, expert) reconstruction term is weighted by g^2, the diagonal of the gated block error, aligning the surrogate with the block outpu…
Downloads · 30 days
16
29% of all-time downloads
All-time downloads
55
Public
Parameters
30.5B
18.5 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors18.5 GB · 100%
How the weights are stored.
U829B · 95%
From the Hugging Face model README
Refinement with a gate-weighted per-expert loss (gate_weight_exponent = 2): each
(token, expert) reconstruction term is weighted by g^2, the diagonal of the gated block
error, aligning the surrogate with the block output the model actually produces rather than an
un-gated equal-sum objective.
Best-performing checkpoint in this series.
| arm | OpenLLM v1 avg | recovery |
|---|---|---|
| dense bf16 | 73.27 | 100% |
| SGPTQ baseline, no refinement | 61.09 | 83.4% |
| refined, p=0 (unweighted) | 64.68 | 88.3% |
| refined, p=2 (this model) | 65.44 | 89.3% |
So the gate weighting is worth +0.76 over the unweighted objective, on top of the +3.6 that refinement itself buys over the one-shot baseline. p=2 wins on all six tasks, led by gsm8k (+2.27).
The p=0 row is the arm re-run on the same code version as this model, which is the comparison that matters. (An earlier revision of this card quoted +1.09 against an older-code p=0 run; that figure was inflated by a code-version difference and has been withdrawn.)
Each exponent was LR-searched independently, so the arms are compared at their own tuned
settings: p=0 at masks_lr 5e-4 / weights_lr 1e-4, p=2 at 2.5e-4 / 5e-5. The lower rates for
p=2 are its tuned optimum, not an uncontrolled difference — the g^2 weighting changes the
effective gradient scale, so the optimal step size moves with it.
Two honest limits on the size of the effect:
paired 4:8 (50%) on routed experts · NVFP4 W4, one FP8 scale per 16 stored elements
(group_size: 32 dense) · NVFP4 A4 dynamic/local · router, attention, embeddings, LM head
untouched · compressed-tensors / nvfp4-pack-quantized
Masks and dense weight values are learned jointly (Gumbel-Softmax over the six legal paired-4:8 patterns, fake-quant forward with STE backward). Scales are never learned: they are recomputed from the current dense weight each forward, detached, and FP8-projected, matching how vLLM computes them at serve time.
Calibration: 512 Hessian samples at seqlen 4096 from 10% neuralmagic/LLM_compression_calibration,
45% open-thoughts/OpenThoughts-114k, 45% HuggingFaceFW/fineweb-edu (sample-10BT).
Refinement ran 10 epochs per layer.
Needs an NVFP4-capable backend (Blackwell-class) and a vLLM build with sparse NVFP4 fused-MoE support. Reasoning benchmarks degrade considerably more than these few-shot tasks on this recipe.