Downloads · 30 days
0
leoncca/Qwen3.8-Flash-Next-NVFP4-QSA-FP8-E4M3-KV-Scales
Qwen3.8-Flash-Next-NVFP4-QSA-FP8-E4M3-KV-Scales is a machine learning model from leoncca. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for vllm. The card lists the license as other.
This repository contains the target's 24 calibrated QSA E4M3 K/V scales and an optional FP8 MTP draft in mtp-fp8/, including its two draft K/V scales. Only the base model needs to be downloaded separately. The target…
Downloads · 30 days
0
Access
Public
Updated Oct 25, 2026
Repo size
5.3 GB
Likes
0
Public
Click a slice to open those files.
.safetensors5.2 GB · 100%
From the Hugging Face model README
This repository contains the target's 24 calibrated QSA E4M3 K/V scales
and an optional FP8 MTP draft in mtp-fp8/, including its two draft K/V scales.
Only the base model needs to be downloaded separately. The target scale file
is 2.64 kB; the optional draft weight file is 5,241,214,400 bytes (4.88 GiB).
This repository is not a standalone text-generation checkpoint.
This is a calibrated 24-tensor K/V scale pack for the main QSA KV cache of:
RadixArk/Qwen3.8-Flash-Next-NVFP47b719225242aacd3dbd3f9407468c2ee9a9d2594model.safetensors.index.json SHA-256:
da5ca9c3b65e48e151329e64e141c2fa700bf2f99aec53cc014e4b52a6ff7a84Do not use these scales with another checkpoint revision. The included materializer checks the base index hash before creating an overlay.
ReVolta supports this checkpoint on
4 x V100 with FP16 activations, E4M3 target KV and the optional FP8 MTP draft.
Follow its checkpoint and draft setup,
quick start and
Standard NVFP4 settings.
Keep VLLM_QWEN4EXP_QSA_E4M3_STRICT_SCALES=1 enabled: missing target scales
then reject startup. Without it the loader warns and uses unit scales;
that fallback is unvalidated and can saturate E4M3 values. Uncalibrated E4M3
MTP draft KV is rejected independently of that target setting.
The calibration and first-phase validation below are historical results with
MTP disabled, FP16 indexer caches and contexts through 128K. Their upstream
runtime source references remain in kvscales-manifest.json. They do not
replace ReVolta's current support scope or its serving acceptance results.
Download the exact base revision and this scale pack:
hf download RadixArk/Qwen3.8-Flash-Next-NVFP4 \
--revision 7b719225242aacd3dbd3f9407468c2ee9a9d2594 \
--local-dir /path/to/base-checkpoint
hf download leoncca/Qwen3.8-Flash-Next-NVFP4-QSA-FP8-E4M3-KV-Scales \
--revision v1.2-base-7b719225 \
--local-dir /path/to/scale-pack
Materialize a derived view. The base directory remains unchanged and must stay available because the derived view uses symlinks for its checkpoint files.
python /path/to/scale-pack/materialize_overlay.py \
--pack-dir /path/to/scale-pack \
--base-checkpoint /path/to/base-checkpoint \
--output-dir /path/to/qsa-e4m3-checkpoint
The tool validates the base revision, the scale-pack checksum, the exact 24
tensor names, FP32 scalar types, positive finite values, and the non-negative-
control marker. It then writes a merged standard model.safetensors.index.json
plus kvscales-provenance.json. A separate scale index is intentionally not
used because the current loader does not discover it automatically.
The optional draft is copied byte for byte from
leoncca/Qwen3.8-Flash-Next-Uncensored-AWQ-g32
revision 1fd7ab705731c9dacc3bcd5b5d15783c60706d64. Its MTP layer was converted
from the Uncensored AWQ checkpoint to FP8 experts; its QSA K/V scales were
calibrated on 2026-09-17. The Base AWQ draft was converted and calibrated
separately and is not the artifact included here. ReVolta validated this
Uncensored draft with this Standard NVFP4 checkpoint, measuring 62.5% MTP
acceptance in that serving test; this is not an answer-quality benchmark.
Keep both mtp-fp8/model.safetensors and
mtp-fp8/model-kvscales-mtp.safetensors. For native context, add this to the
ReVolta launch command using the materialized target view and
--quantization modelopt_mixed --kv-cache-dtype fp8_e4m3:
--speculative-config '{"method":"mtp","model":"/path/to/scale-pack/mtp-fp8","quantization":"modelopt_mixed","num_speculative_tokens":4,"draft_sample_method":"greedy","attention_backend":"FLASH_ATTN_V100","kv_cache_dtype":"fp8_e4m3"}'
For 512K context, first materialize the target and then make YaRN views for
both target and draft, as shown in the ReVolta README. Set the draft model
to that draft view. Use "kv_cache_dtype":"float16" for an FP16 draft cache.
The draft's own README, LICENSE, provenance, validation and checksum files
are preserved unchanged. That README describes its original matching AWQ
target; the cross-checkpoint NVFP4 pairing is documented here.
K/V activations were collected with FP16 KV through normal inference forwards after engine initialization. Dummy/profile/graph-capture forwards were excluded. The calibration distribution covered Chinese, English, code, multi-turn dialogue, tool use, and 1K/16K/64K/128K contexts. Calibration and quality inputs were disjoint. Raw private tool-use records are not distributed.
The converter contract is:
stored = e4m3fn(x / scale)
reconstructed = stored * scale
scale = observed_max_envelope / 448
The 24 FP32 scales range from 0.0171247218 to 0.0841238871. No saturation
was observed for any K/V tensor. Per-layer max-abs, p99.9, p99.99, saturation,
and shard-stability summaries are recorded in kvscales-manifest.json.
1.836x).0.990783, FP16 recall 0.995301,
and micro Jaccard 0.987579.0.999311, relative L2 0.0324132, RMSE 0.0160753.FP16 XQA used P1024 while generic E4M3 G6 XQA used P256, so the full performance A/B includes reduction order as a second variable. The quality run did not enter XQA and was not affected by this difference.
mtp-fp8/LICENSE.
Follow the base and source model license terms.The authoritative scale file is model-kvscales.safetensors:
bbf767cd46fe3ac52793c87ec069a964bf6fb64c5b6183959c734b9772936cc9
See SHA256SUMS for the complete release manifest.