Mage-Flow-Turbo-XPO3-NVFP4

<p align="center"> <img src="xpo3-nvfp4-icon.png" width="360" alt="XPO3 NVFP4 speed mascot"> </p>

Mage-Flow Turbo NVFP4 — V2

Follow Mage-Flow NVFP4 development, benchmarks, and release updates at Arands.com.

Speed and quality through cross-precision optimization.

AJH Mage-Flow releases

Only the Balanced transformer policy was packaged for Turbo before XPO3. Fast and Quality are non-Turbo Mage-Flow releases.

This is the V2 runtime release for the existing model files. XPO3 is the name of the bundled runtime profile, not a new model format or loader. With the current ComfyUI-XPO3 v0.3.0 custom node, the weights run through the promoted XPO3 runtime stack: a four-step, ComfyUI-oriented derivative of microsoft/Mage-Flow-Turbo optimized for native Blackwell NVFP4 execution. The additional speed is a runtime update and does not require rebuilding or redownloading the model weights.

Recommended AJH Mage-Flow Turbo release. In the reviewed general-prompt set, XPO3 follows the original BF16 Turbo closely while delivering the fastest validated runtime in this project. Individual trajectories can occasionally improve a detail as well: the published golden-retriever comparison avoids the duplicate-tail artifact present in its BF16 counterpart. That is an observed example, not a claim that XPO3 is universally better than BF16.

Microsoft reports 0.59 s/image at 1024×1024 on one A100 for the original Mage-Flow-Turbo, at roughly 18–20 GB peak memory. This V2 runtime measured 0.5943 s median at 1024×1024 on an RTX 5080 with 11.85 GiB peak allocated. These are comparable latency points, not an apples-to-apples cross-hardware benchmark. See the upstream model card.

Three precisions. One fused runtime.

NVFP4 supplies transformer speed, scaled FP8 compresses the Qwen3-VL text encoder, and BF16 is retained only where matched testing found that four-bit execution caused visible quality damage.

Native W4A4 runtime—not a weight-only dump

XPO3 runs its quantized transformer projections through the native Blackwell W4A4 path: weights are stored as block-16 NVFP4, BF16 activations are dynamically quantized to NVFP4 with role-specific scaling, and the matrix multiplication executes as native NVFP4. It does not expand the packed weights to BF16 and fall back to a BF16 GEMM.

That fast path is one part of the promoted measured runtime stack:

• 44 MLP projections and 24 offline-fused QKV groups use native W4A4;
• four quality-sensitive late image MLP projections remain BF16;
• 23 eligible up projections use fused bias-plus-GELU NVFP4 execution;
• calibrated FP4 image-MLP bridge routing is enabled on blocks 0-8;
• official Sparge/Sage2 routing runs on steps 1-2 across all 12 blocks;
• the CFG1, batch-one direct-HND processor is used on that routed path;
• steps 0 and 3, disabled toggles, and unsupported envelopes stay on the packaged exact fallback path;
• the complete pipeline stays resident, while prompt conditioning and the fixed policy prefix are cached.

The conditioning cosines describe text-encoder outputs, not an overall image quality score. Image quality was also checked with frozen same-seed prompt suites and direct visual review.

Download

This repository intentionally contains the three model files useful to ComfyUI:

The diffusion model is one file rather than four Hugging Face transformer shards. The complete three-weight download is 10.45 GB.

Install ComfyUI-XPO3, restart ComfyUI, and use:

1. Load Mage-Flow Turbo XPO3
2. Mage-Flow Turbo XPO3 Generate

The custom-node repository includes an API-format starter workflow that defaults to 1280×1280. Recommended settings are four steps, CFG 1, and static shift 6.

The Generate node also exposes independent runtime toggles for fused GELU-up, the FP4 image-MLP bridge, bridge-block selection, Sparge/Sage2 attention, direct-HND, and selected attention steps and blocks.

V1 → V2

No model-weight redownload is required. V2 is a runtime and integration upgrade: fused GELU-up, the calibrated FP4 bridge, the validated accelerated attention/direct-HND route, explicit per-feature fallbacks, a 1280-default ComfyUI workflow, and a reproducible benchmark runner in ComfyUI-XPO3.

Speed: 0.59 s at 1024, 0.96 s at 1280 on RTX 5080

Measured on 16 GB RTX 5080 and RTX 5060 Ti cards under Linux, Python 3.11, and PyTorch 2.13.0+cu130, with one resident model load per matrix arm. The table distinguishes repeated interactive generation from the first image of a new prompt:

• Hot: model remains resident; prompt conditioning and its fixed screening prefix are reusable.
• Fresh prompt: model remains resident, but the prompt-conditioning and screening caches begin empty. This is not a cold process start.

Prompt conditioning is the Qwen3-VL embedding pass used by the diffusion model; it is independent of the random seed and can therefore be reused for re-rolls. The screening pass checks a prompt against a fixed content-policy prefix. Its immutable prefix KV state is also cacheable and does not change the generated image.

The 1024 and 1280 hot figures are V2 resident runs. The 0.9570 s 1280 figure is the more robust three-prompt × three-seed matrix median; the 1024 figure is a current V2 two-seed repeat. In the completed multi-prompt matrix, the promoted stack delivered a consistent 1.177x-1.212x end-to-end gain over the previous packaged resident FA2 baseline.

For process startup on the RTX 5080, model load plus resident placement takes about 9.46-10.14 s before the first generation. That one-time cost is kept separate from the generation timings above.

Reproduce the hot/fresh-prompt protocol

The matching public runner ships with ComfyUI-XPO3. From the plugin directory, using the same ComfyUI Python environment and model filenames:

python benchmark.py --width 1024 --height 1024 \
  --output xpo3_1024_benchmark.json

It runs a fixed three-prompt/three-seed suite at four steps, CFG 1, and static shift 6; records a fresh-cache warm-up and repeated hot generations for each prompt; and writes timing, environment, and active-feature evidence to JSON. Your GPU, driver, and ComfyUI environment will produce different numbers.

Peak allocated memory increased modestly versus that packaged baseline:

• 1024×1024: 11.85 GiB peak allocated (about +51.0 MiB)
• 1280×1280: 12.33 GiB peak allocated (about +78.6 MiB)

The complete promoted stack still remained under 16 GB on both validated cards.

Timings are hardware- and environment-dependent.

Promoted runtime stack and fallback rules

The current v0.3.0 custom-node runtime includes:

• fused GELU-up on eligible NVFP4 up projections;
• calibrated FP4 image-MLP bridge on image blocks 0-8;
• official Sparge/Sage2 meansim top-k 1.0, smooth-K, all 12 blocks on steps 1-2;
• the validated CFG1, batch-one direct-HND processor on that routed path;
• exact packaged fallback on steps 0 and 3;
• independent per-feature toggles exposed through the ComfyUI generate node.

Fallback behavior is intentional rather than best-effort:

• when fused GELU-up is disabled, the wrapper calls the exact original activation module;
• when the bridge is disabled, a bridge block is deselected, or an unsupported shape/stream/device condition is seen, that block falls back before native launch;
• when attention acceleration is disabled, CFG is not 1.0, steps are not 4, the pipe is unsupported, or the Sparge dependency is unavailable, the packaged attention path remains active;
• direct-HND is only used on the supported CFG1, batch-one routed envelope.

Quality

XPO3 does not quantize every eligible projection merely to maximize the compression ratio. This Turbo release ships one selected Balanced policy. The Fast and Quality rows below are research controls used to choose it; they are not additional downloadable Turbo profiles.

1280×1280 BF16 versus XPO3 comparison set

Each pair below uses the same prompt, seed, 1280×1280 resolution, four steps, CFG 1, and static shift 6. Original Mage-Flow-Turbo BF16 is on the left; XPO3 NVFP4 is on the right. The images are direct decodes with no upscaling or postprocessing. Click any comparison to inspect it at full size.

Across this general-prompt set, XPO3 closely follows BF16 subject, framing, lighting, palette, and overall quality. Small trajectory-dependent differences remain, so XPO3 should not be interpreted as pixel-identical or lossless.

At 1024×1024 in the precision-policy study:

That study isolated transformer precision before the later resident, single-branch, QKV-fusion, cache, and fused-GELU speed work, so its latency numbers should not be mixed with the final warm-runtime table above.

Representative 1280×1280 XPO3 output

This is a direct, unprocessed four-step XPO3 decode: 1280×1280, CFG 1, static shift 6, and seed 3334910515.

A 4K resolution high detail photo realistic image of the top half of a cyborg woman with dark black hair, striking blue eyes that have a very subtle glow in the iris, standing side profile, head tilted up towards the sky with a questioning expression, she has subtle gaps in her skin that hint at a robotic nature, outdoor forest night setting, sky filled with bright brilliant stars that glow against the dark setting, nebula visible

This intentionally difficult prompt combines photorealistic anatomy, exposed machinery, damaged skin, glowing eyes, and complex night lighting. It is shown as one representative current result, not as evidence that every prompt is pixel-identical to BF16.

What XPO3 changed

1. Architecture-aware precision selection

Forty-four Mage transformer MLP projections run in native NVFP4. Four late image-stream projections remain BF16:

• transformer_blocks.9.img_mlp.net.2
• transformer_blocks.10.img_mlp.net.2
• transformer_blocks.11.img_mlp.net.0.proj
• transformer_blocks.11.img_mlp.net.2

Keeping these four projections in BF16 substantially improved the 1024 portrait result while adding far less size and latency than returning the complete image stream to BF16.

2. Quality-selected activation scaling

BF16 activations are dynamically quantized to block-16 NVFP4 immediately before native W4A4 execution. Scaling is selected by projection role: the NVFP4 MLP down paths use a 0.75 activation-scale multiplier, while the other native projections retain their validated scaling policy. Alternative block-scale searches, weight 4-over-6, and bias-correction candidates were tested and rejected because lower local reconstruction error did not consistently produce a better final diffusion trajectory.

3. Offline-fused NVFP4 attention

Each image/text Q, K, and V triplet is concatenated into one native NVFP4 projection. All 24 fused QKV groups are packed in the downloadable checkpoint:

• 72 original BF16 Q/K/V projections replaced;
• approximately 0.91 GiB lower peak allocation in the experimental matched attention test;
• no 6.29-second load-time attention packing step;
• no full-weight BF16 expansion or BF16 GEMM fallback for these projections.

4. Fused MLP execution

Twenty-three NVFP4 MLP up projections use a native cuBLASLt up-projection-plus-bias-plus-GELU path. The final quality-sensitive BF16 image up projection remains unfused, and the fused path remains independently toggleable from the ComfyUI node.

5. Calibrated image-MLP bridge

The promoted runtime adds the calibrated FP4 image-MLP bridge on image blocks 0-8, using the pinned per-block scales selected during the bounded bridge validation. The bridge is independently toggleable and retains the packaged module as its exact off path.

6. Step-gated Sparge/Sage2 attention

The promoted runtime adds official Sparge/Sage2 routing on steps 1-2 across all 12 blocks, with the validated CFG1, batch-one direct-HND processor on that route. Steps 0 and 3 remain on the packaged exact path, and the attention route is independently toggleable.

7. Resident and cached generation

The transformer, FP8 text encoder, and VAE fit together on the validated 16 GB GPU. CFG 1 uses a single conditional branch. Exact prompt conditioning is cached independently of the random seed, and the fixed policy-screen prefix uses a bounded KV cache.

8. Single-file deployment

The XPO3 diffusion checkpoint embeds the transformer structure, precision policy, fused-attention mapping, native-runtime requirement, and base-model relationship in safetensors metadata. It contains 437 tensors and is approximately 977 MB smaller than the earlier four-shard Balanced transformer.

What XPO3 is—and is not

XPO3 is a checkpoint and runtime approach focused on speed and quality through architecture-aware cross-precision deployment and fused native execution. It uses NVIDIA's standard NVFP4 E2M1 block-16 representation; it does not claim to invent a new floating-point datatype.

It is also not SVDQuant or a Nunchaku checkpoint. XPO3 does not use a high-precision low-rank outlier branch. Its quality recovery comes from measured precision placement and scale policy, while its speed comes from native NVFP4 execution, projection fusion, epilogue fusion, single-branch Turbo inference, and resident caching.

Requirements and current limitations

• Linux x86-64
• NVIDIA Blackwell SM120
• under 16 GB on the validated promoted stack; 16 GB class validated
• Python 3.11
• PyTorch 2.13.0+cu130
• TorchVision 0.28.0+cu130
• TorchAudio 2.11.0+cu130
• CUDA 13.x runtime; bundled binaries built with CUDA 13.1
• flash-attn==2.8.3

The first prebuilt runtime does not support Windows, WSL, pre-Blackwell GPUs, Python 3.12/3.13, or other PyTorch/CUDA ABIs.

The custom-node plugin also bundles SpargeAttn runtime code and extensions under runtime/spas_sage_attn/ under Apache-2.0 terms. Its included license text is at third_party_licenses/SpargeAttn-LICENSE in the plugin repository.

The files use ComfyUI's normal diffusion-model, text-encoder, and VAE directories. Mage-Flow is not currently supported by ComfyUI's generic diffusion loader, so this release requires the XPO3 custom nodes and does not yet connect directly to stock KSampler, LoRA, ControlNet, or native Comfy conditioning nodes.

Image-quality limitations

Four-step Turbo is intentionally aggressive. Complex photorealistic prompts that combine organic anatomy with exposed mechanics—such as the cyborg stress prompt in this card—can produce seed-sensitive neck, shoulder, torso, or small-detail defects. A matched ten-seed comparison found the same failure class in the original all-BF16 Mage-Flow-Turbo model; it is not an XPO3-only defect.

XPO3 can change the diffusion trajectory, so it is not pixel-identical or lossless. The published 1280 BF16/XPO3 gallery is intended to make that trade off inspectable rather than hidden.

Sources and licenses

• Mage-Flow-Turbo, the VAE, and the Mage inference source derive from microsoft/Mage-Flow-Turbo under the upstream MIT terms.
• The scaled-FP8 Qwen3-VL checkpoint comes from starsfriday/Mage-Flow-Edit-FP8 and derives from Apache-2.0 Qwen3-VL weights.
• Bundled SpargeAttn runtime code and extensions under runtime/spas_sage_attn/ are Apache-2.0; see thu-ml/SpargeAttn and the plugin's third_party_licenses/SpargeAttn-LICENSE.
• XPO3 custom-node and packaging code is MIT licensed.

See the ComfyUI-XPO3 third-party notices and included license texts in the custom-node repository.