Qwen-AgentWorld-35B-A3B-HybridQuant-NVFP4

Qwen-AgentWorld-35B-A3B — HybridQuant NVFP4 + MTP)

Mixed-precision NVFP4 quantization of Qwen/Qwen-AgentWorld-35B-A3B for the NVIDIA GB10 (DGX Spark, sm_121a) via vLLM's modelopt_fp4 path. Built with our "HybridQuant" recipe.

• Performance 76.7 TPS w/o MTP, 110.9 TPS with MTP
• Experimental This model is experimental. Nothing is guarantee except possibly a waste of your time. If it doesn't work well, tell us. If it works well, tell everybody.

Recipe

quant_algo: MIXED_PRECISION (NVIDIA TensorRT Model Optimizer, w4a16_nvfp4_mse-fp8_attn-kv_fp8_cast):

• NVFP4 W4A16 (group 16, fp8 scales, MSE fp8-scale-sweep calibration) on MoE experts, shared-expert MLP, lm_head
• FP8 attention / linear-attention projections · FP8 KV cache
• Norms, embeddings, router at BF16; visual tower left BF16 in this build
• MTP drafter: Measured draft acceptance: 85-99%.

Measured (GB10 / DGX Spark, vLLM eugr nightly 0.23.1)

Footprint ≈ 22.8 GiB (incl. 1.57 GiB MTP shard).

Validation status — read before relying on this model

• No BF16 A/B reference YET: provisionally validated.
• Benched with the same harness/settings (qwen3_coder tool parser, thinking off, temp 0, prefix caching off).
• Per-category bench scores carry a ±1-scenario noise floor; category swings between runs are noise, totals are signal.

Serving (vLLM)

vllm serve JasonW2025/Qwen-AgentWorld-35B-A3B-HybridQuant-NVFP4 \
  --quantization modelopt_fp4 --kv-cache-dtype fp8 --calculate-kv-scales \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}' \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder --reasoning-parser qwen3 \
  --default-chat-template-kwargs '{"enable_thinking": false}' \
  --trust-remote-code --max-model-len 131072

Provenance & license

Derivative of Qwen/Qwen-AgentWorld-35B-A3B (and MTP tensors from Qwen/Qwen3.6-35B-A3B); the original models' licenses and terms apply. Quantized with NVIDIA TensorRT Model Optimizer. Not affiliated with or endorsed by Qwen/Alibaba or NVIDIA.