Qwen3.8-27B-QAT-NVFP4

Qwen3.8-27B-QUASAR-NVFP4-mlx

MLX conversion of QUASAR-QAT/Qwen3.8-27B-QUASAR-NVFP4, a 4-bit NVFP4 (W4A4) quantization-aware-trained build of Qwen/Qwen3.8-27B where all 496 linear layers (self-attention, gated delta-net, and MLP) are NVFP4. Converted from the source compressed-tensors nvfp4-pack-quantized checkpoint to the MLX-native nvfp4 layout for the omlx / mlx-vlm runtime on Apple silicon.

• ~19.15 GiB: 4 weight shards (18.36 GiB) + mtp.safetensors (MTP draft head, 15 tensors, 810 MiB).
• 1695 tensors: 496 packed NVFP4 weights (uint32) + 496 E4M3 scales (uint8)
  • 703 BF16 (embeddings, lm_head, vision tower, norms, conv1d, A_log, dt_bias)
  • the 15-tensor MTP head.

Conversion notes

MLX's nvfp4 kernel is single-level and does not carry the per-tensor global scale, so the two-level source scaling is folded into the per-group E4M3 scales: the E2M1 codes are kept bit-exact and each per-group scale is stored as E4M3(decode(weight_scale) / weight_global_scale). This single re-rounding of the (much smaller) scale tensor is the only precision change versus the source; the packed 4-bit weights themselves are byte-identical.

MTP norm weights carry the MLX +1.0 RMSNorm shift; the MTP head is a separate mtp.safetensors side file.

Produced with convert_vllm_nvfp4_to_mlx.py.