Downloads · 30 days
280
31% of all-time downloads
lambsea/Qwen3.8-Flash-Next-Uncensored-NVFP4-AWQ-GPTQ
Qwen3.8-Flash-Next-Uncensored-NVFP4-AWQ-GPTQ is a text generation model from lambsea. Use it when you need the model to write or continue text. It is set up for vLLM. The card lists the license as apache-2.0.
Calibrated 4-bit weights of orcarouter/Qwen3.8-Flash-Next-Uncensored: routed experts in NVFP4 (W4A4, AWQ smoothing + GPTQ error compensation on ~1K calibration sequences), attention / GDN / shared-expert in FP8 per-ou…
Downloads · 30 days
280
31% of all-time downloads
All-time downloads
915
Public
Parameters
180B
184 GB on disk
Likes
2
Public
Click a slice to open those files.
.safetensors183 GB · 100%
How the weights are stored.
U8121B · 67%
From the Hugging Face model README
Calibrated 4-bit weights of orcarouter/Qwen3.8-Flash-Next-Uncensored: routed experts in NVFP4
(W4A4, AWQ smoothing + GPTQ error compensation on ~1K calibration
sequences), attention / GDN / shared-expert in FP8 per-output-channel;
embeddings, lm_head, norms, router gates and the PLE table kept in BF16.
Includes the MTP head (model-mtp.safetensors).
Measured against the BF16 base model on 328,800 positions of self-distilled thinking-mode output (8 documents: 2 multi-turn agentic coding sessions with tool loops, 6 plain reasoning/math/writing seeds, all generated to natural stop), teacher-forced top-1/top-2 at every position. 8xRTX Pro 6000, TP8 + expert parallel, vLLM 0.29.0, BF16 KV (see Serving notes). Agreement = raw argmax match vs the BF16 base; buckets split by the BF16 top1-top2 logprob margin. Disagreement % shown (lower is better).
| checkpoint | top-1 | near-tie | moderate | confident | certain |
|---|---|---|---|---|---|
| orcarouter community NVFP4 | 93.90% | 39.82 | 12.34 | 2.60 | 0.15 |
| this model (NVFP4 + AWQ + GPTQ) | 93.93% | 39.67 | 12.31 | 2.59 | 0.12 |
Per-category disagreement vs BF16 (tokens scored):
| category | tokens | this model | community NVFP4 |
|---|---|---|---|
| coding (agentic, tool loops) | 200,682 | 7.76% | 7.79% |
| math | 53,166 | 3.98% | 4.05% |
| reasoning | 63,769 | 2.39% | 2.38% |
| writing | 11,175 | 6.67% | 6.95% |
Paired per-position logprob test on the BF16-preferred token: this model carries +0.0019 nats over the community quant (SE 0.0005, t=+4.1, 321,736 paired positions). Residual disagreement concentrates in BF16 near-ties (whitespace width, table digits, synonym flips) — sampled splits show no reasoning or code defects on either side.
Tested with vLLM >= 0.29 (native qwen4_exp support), TP8 with
--enable-expert-parallel on 8xRTX Pro 6000 (tree is ~180 GB):
vllm serve lambsea/Qwen3.8-Flash-Next-Uncensored-NVFP4-AWQ-GPTQ \
--tensor-parallel-size 8 --trust-remote-code \
--enable-expert-parallel --enable-auto-tool-choice \
--tool-call-parser qwen3_coder
Notes:
Qwen4Exp QSA does not support KV quantization. The
shipped config.json declares no kv_cache_scheme.model.language_model.layers.*
prefix (VLM-wrapper naming); the checkpoint loads through the
Qwen4ExpForConditionalGeneration path even for text-only serving, so
preprocessor_config.json must stay in the repo.calibration/ (see its README) — no serving path reads them today;
they exist for future static-KV/W4A4 work.weight_scale of shape
[out_features, 1]; NVFP4 experts carry block-16 weight_scale
(F8_E4M3, in global-scaled space) plus a quant-side scalar
weight_global_scale — i.e. the kernel dequant multiplier is
1 / weight_global_scale, the standard checkpoint convention.Quantized 2026-09 on 8xH100 (8-GPU DDP calibration, compressed-tensors disk-offload pipeline), exported to a serving tree, repaired scales (FP8 per-channel re-quantized from the linearized BF16 source; NVFP4 global scales normalized to the quant-side convention), and validated end-to-end by the benchmark above. All measurements, corpus, and the BF16 ground-truth score files are reproducible from the published run profile.