Qwen3.8-Flash-Next-NVFP4

Qwen3.8-Flash-Next-NVFP4

GSM8K unchanged at 97.27 and AIME26 majority@8 unchanged at 100, while a decode step at low concurrency reads 38% fewer weight bytes — 6.13 GB/token against the base's 9.95 GB.

Designed for single-device DGX Spark serving, where the PLE n-gram tables stream from SSD and the dense path is what a decode step spends its bandwidth on. DGX Spark inference results coming soon; the numbers on this card were measured on 2xB200.

Derived from RadixArk/Qwen3.8-Flash-Next-NVFP4, which quantizes the routed MoE experts to NVFP4 and leaves everything else in BF16. The dense path it left in BF16 — attention, the gated-delta-net projections, the shared experts — is what a low-concurrency decode step spends most of its bandwidth on.

This checkpoint takes that dense path to FP8 W8A8 (static per-tensor, with calibrated activation scales) and the LM head to NVFP4 W4A16. Routed experts and the PLE n-gram tables are byte-identical to the base.

301 tensors are requantized; 1,562 unchanged tensors are byte-identical to the base.

What changed

Checkpoint size: 131.4 GB (base 135.2 GB).

Activation scales were calibrated by max over MoE-block input activations captured from live serving; the LM head is weight-only and needs no activation scale.

Decode weight traffic

Bytes of weights read per generated token, computed from the real tensor shapes (top-10 of 512 routed experts active):

At low concurrency the dense path dominates what a decode step reads, and the dense path is exactly what this checkpoint quantizes — the routed experts, already NVFP4 in the base, are left alone.

Evaluation

Base column is the number published on the base model's card. This checkpoint was measured on SGLang, TP2, 2xB200, using the base card's protocols verbatim — GSM8K full 1319 at t0.6 / top-p 0.95 / max 8192, AIME26 30 problems x 8 at t1.0 / max_tokens 130000, both with the NeMo-Skills boxed prompt.

Usage

The architecture (Qwen4ExpForConditionalGeneration) is not yet in mainline SGLang. Build from PR #36497, which is sgl-project's own branch — no third-party patch is needed.

python -m sglang.launch_server \
  --model-path senfu/Qwen3.8-Flash-Next-NVFP4 \
  --tp 2 \
  --quantization modelopt_mixed \
  --fp4-gemm-backend flashinfer_cutlass \
  --moe-runner-backend flashinfer_trtllm \
  --page-size 64 \
  --mamba-scheduler-strategy extra_buffer \
  --mamba-track-interval 64 \
  --chunked-prefill-size 4096 \
  --max-running-requests 36 \
  --context-length 262144 \
  --mem-fraction-static 0.80 \
  --allow-auto-truncate \
  --port 30000

--quantization modelopt_mixed is the one flag that differs from the base's recipe: this checkpoint declares ModelOpt MIXED_PRECISION, with a per-module quantized_layers map covering 396 FP8 modules, 48 NVFP4 MoE layers and 1 W4A16 head.

Keep --mamba-scheduler-strategy extra_buffer: it is what makes --page-size 64 legal for the GDN cache, and page 64 is required by compressed-QSA addressing.

License

Qwen Community License 1.0, inherited from the base model. Derivative works are permitted; the license and copyright notice must travel with them. Operating a Model-as-a-Service or AI Work Assistant business on it requires a separate agreement with Qwen.

Credits

Qwen for the base model. RadixArk for the NVFP4 expert quantization this builds on. SGLang for the serving stack.