Image-2.1-Calibrated-NVFP4

Image 2.1 Calibrated NVFP4

Built with Qwen. A calibrated NVFP4 transformer derived from Qwen/Qwen-Image-2.1 at revision b3179ad355be050328e483a9dfdd9e60cd62adfa.

4.87 GB transformer; native SM120 W4A4 Tensor Core kernels; all 40 denoising steps. Encoder, VAE, attention, conditioning and small projections retain BF16. Each of 224 large projections uses NVFP4 plus a BF16 rank-128 correction, accumulated together in FP32. This is a custom Diffusers/FlashInfer format and requires the included loader.

Measured speed

RTX PRO 6000 Blackwell Workstation 96 GB, batch 1, CFG 1, prefix KV cache. Mean CUDA-synchronized wall time includes text encoding, all denoising and VAE decoding. Loading, one full warmup per resolution, compilation and PNG writing are excluded. NVFP4 uses five measured 1024 runs and three 2048 runs; fresh FP8 controls use two per resolution in the same clean environment.

Earlier BF16 measurements on this workstation with the same generation settings were 9.792 and 55.169 seconds respectively. These are historical controls from the same project, not newly timed in this release. First-use loading and compilation cost extra; full warmup times are in benchmark.json. Performance depends on resolution and prompt. No steps are skipped, no approximate feature reuse is used, and attention is not quantized.

The 30-second real-time video has no text, overlays or audio. It begins with one completed warmup image, then displays actual newly completed generations with all waits preserved. The capture log records every display update and frame timestamp.

Quality and calibration

Calibration uses 64 original BF16 trajectories: 56 text-to-image and eight edits, with 1024, 2048 and alternate aspect ratios. Each projection has 1,536 sampled activation rows across six denoising timesteps; 384 stratified fit rows and 384 disjoint diagnostic rows. Five smoothing exponents and two weight-scale methods are searched using native FP4 kernels. Weight-scale refinement searches 15 per-block scale factors, weighted by activation second moments. Rank-128 SVD corrections absorb dominant weight components. Original BF16 weights are the source; this is not a requantization of FP8.

Activations use a fresh actual tensor-wide amax on every call, followed by dynamic E4M3 scales per 16 values. Sparse calibration alone missed rare large activations; static activation ranges caused clipping and visible texture degradation, so that candidate was rejected. The final runtime recalibrates and evaluates with dynamic scaling. BF16 correction weights are rescaled consistently when the activation global scale changes.

The 18-case development validation set includes 16 generation prompts and two edits. Against BF16, mean LPIPS(AlexNet,512px) is 0.122970, SSIM 0.897363, and full-latent cosine 0.976903. Eight additional prompts were frozen after candidate selection; their mean LPIPS is 0.124012 and SSIM 0.871713. Full images, paired comparisons and individual measurements are included.

This is lossy quantization, not a zero-quality-loss guarantee. All 26 pairs were visually reviewed. The corrected candidate preserves readable primary English/Chinese text, transparency, image editing, fine felt/fur textures and plausible hands in these tests. Same-seed composition, poses, decorative marks, geometry and fine detail can change; the largest validation differences are the pottery and train scenes. The prior calibrated FP8 release is closer to BF16 numerically (original 18-case mean LPIPS 0.03447). Fidelity metrics do not establish broad human preference or guarantee every prompt.

Native FP4 evidence

An actual denoising-step profile records 224 SM120 block-scaled f4E2M1FN GEMM launches, 32 native BF16 FlashAttention launches and 40 transformer calls per 40-step generation. Packed E2M1 values and E4M3 scale layouts were independently decoded and checked against FP32 matrix products. See kernel evidence and numerical verification.

The engine uses FlashInfer NVFP4 SVDQuant, pinned to commit 975f90583d9ac8896db14cf0f26e99a853c2f136. PyPI 0.6.18 lacked the SM120 fusion shown in the online documentation during development, so use the pinned source revision. Generic Transformers FP8/FP4 loading does not select this custom runtime.

Install and generate

Tested on Linux x86_64, Python 3.12, Torch 2.14.0+cu130, CUDA toolkit 13.3, driver 615.71.09, compute capability 12.0. Other hardware is unverified. Download this repository, then run bash install.sh from its directory. The script installs pinned dependencies and the pinned FlashInfer source; a recent NVIDIA driver and CUDA toolkit must already be available.

.venv/bin/python generate.py --prompt 'A kingfisher on a mossy branch, detailed feathers, no text' --width 1024 --height 1024 --steps 40 --warmup --output kingfisher.png

--base /path/to/local/original-model reuses a local upstream checkpoint. Otherwise the loader downloads the pinned upstream components. --image input.png enables editing. --prompts-json prompts.json accepts a JSON list and reuses the loaded pipeline. --eager disables block compilation. Warmup and loading costs are reported separately; without warmup the reported generation time includes any compilation on that call.

from nvfp4_runtime import load_pipeline
from acceleration import accelerate_pipeline
pipe = accelerate_pipeline(load_pipeline('Qwen/Qwen-Image-2.1', './transformer'))
image = pipe(prompt='A detailed watercolor garden', width=1024, height=1024, num_inference_steps=40).images[0]
image.save('garden.png')

Do not cast the loaded quantized transformer with .to(dtype=...); packed values and FP32 scales must retain their stored types. The normal loader handles device placement. The checkpoint is not a generic Transformers, ComfyUI, TensorRT or Nunchaku checkpoint.

Reproduce calibration

Download the exact BF16 upstream revision to a local directory first. The generation runtime environment also supports calibration. From this release directory:

export IMAGE21_BASE=/absolute/path/to/original-model
export IMAGE21_WORK_DIR="$PWD/rebuild"
export PYTHONPATH="$PWD"
.venv/bin/python source/collect_calibration.py
.venv/bin/python source/quantize_dynamic.py

The first command collects the published 64-case calibration manifest; the second reconstructs the NVFP4 transformer under rebuild/release/transformer. Original fitting activations are retained on the build workstation and can be regenerated with these scripts. Evaluation additionally needs LPIPS, scikit-image and SciPy; video capture uses imageio-ffmpeg. Environment versions, source, calibration search, manifests and timing records are included. Earlier static-range, mixed-precision, ridge-correction, kernel-autotuning and CUDA-graph experiments were rejected or provided no useful gain; they are not enabled in this runtime.

License and modification notice

Built with Qwen. ProCreations modified the original transformer through calibrated NVFP4 quantization and supplied this custom inference runtime on 2026-09-20. This derivative is distributed under the included Qwen Research License and Notice, for non-commercial research/evaluation. Preserve those files and the upstream terms when redistributing. This is an independent derivative, not an official Qwen release.