Qwen3.8-Flash-Next-mixed-NVFP4-FP8-PLE-FP8

Qwen3.8-Flash-Next-mixed-NVFP4-FP8 + FP8 per-row PLE table

One repo id for a configuration that otherwise takes two. Contents:

• primitive-ai/Qwen3.8-Flash-Next-mixed-NVFP4-FP8 at fd7dbb007b83d4ac41db93ccd1cf2b352c6e8bd2, minus its 43 ple-bf16-*.safetensors shards, with model.safetensors.index.json trimmed to match (296630 -> 296502 tensors).
• ples_fp8/ -- the FP8 per-row quantised n-gram table (48.9 GiB) from primitive-ai/Qwen3.8-Flash-Next-PLE-quant at 4fb644639dc2a5abcf34100c0a3c45fd2a00ba29.
• The three overlay files from that same repo: worker_image_quant.py, ple_layer_quant.py, connector_mrv2.py.

Nothing is re-quantised or otherwise modified; every file is a byte-identical copy of one of the two sources.

Why drop the BF16 table

The 43 dropped shards hold exactly 128 tensors, all of them ...ple.ple_embedding.ngram_embedding.shard_N.weight -- the 95.4 GiB BF16 n-gram table. Served with the quantised sidecar in ples_fp8/, nothing reads it: GPU workers skip PLE layer init under VLLM_PLE_CPU_OFFLOAD, and the offload worker filters those tensors out of the checkpoint stream once VLLM_PLE_QUANT_DIR is set. vLLM still fetches every *.safetensors in a repo before it loads anything, so carrying them means downloading and storing 95.4 GiB in order to ignore it.

Use the upstream repos instead if you want the BF16 table -- for the in-RAM path (~100 GB of host RAM) or the BF16-on-NVMe overlay. This packaging only makes sense with the quantised table.

Serve

Mount the three overlays over the image's site-packages and point VLLM_PLE_QUANT_DIR at ples_fp8/ inside the snapshot. See the PLE-quant card for the full command and for primitive-ai's measurements.

VLLM_GDN_DECODE_KERNEL=triton is required by this build's FP8 GDN projections; the default cuda kernel stalls under concurrency.

All credit for both the checkpoint and the tables to primitive-ai; this is a repackaging. Weights under Qwen Community 1.0, tables under Apache-2.0.