Image-2.1-Calibrated-FP8

Image 2.1 Calibrated FP8

Built with Qwen. An independently calibrated W8A8 E4M3 quantization of the image transformer from Qwen-Image-2.1. Research/evaluation only under the included Qwen Research License.

This repository contains the quantized transformer, an explicit inference loader, reproducible calibration/evaluation code, comparisons, profiler evidence, and a real-time 30-second image-only demonstration. The original text encoder, VAE, processor and scheduler are loaded from the pinned upstream model. The encoder and VAE remain BF16.

Native transparent PNGs and exact prompt reuse

The FP8 model natively generates RGBA images. --transparent adds a transparency instruction to your prompt and requires a PNG output, preserving the original alpha channel. It does not remove backgrounds after generation or change the model, VAE, denoising steps or precision. Transparency remains prompt-dependent; the CLI reports alpha statistics.

python generate.py --prompt "A cute mint-green baby dragon mascot, full body" --transparent --width 1024 --height 1024 --steps 40 --warmup --output dragon.png

For several variations of the same text prompt in one loaded process, --cache-prompts reuses the exact encoder outputs. It holds at most eight prompts, never caches image edits, and does not reuse denoising features. New prompts still run the encoder. Six paired checks, including 2048 output and two transparent-image edits, produced bit-identical final latents and RGBA pixels versus the existing optimized FP8 runtime.

Warmed batch 1, all 40 steps, CFG 1, same calibrated FP8 weights and BF16 attention/encoder/VAE on the RTX PRO 6000. Five interleaved measurements per mode at 1024 and three at 2048. Times include the complete generation path; cached mode omits the repeated encoder computation on cache hits. Load, compilation/warmup and PNG writing are excluded. These timings show no material general speedup. This does not reach one image per second, and cached timings do not apply to new prompts or edits.

Transparent corgi PNG · Transparent dragon PNG · 2048 watercolor PNG · Measurements and further optimization trials.

Checkerboard previews below are display aids. The linked PNGs retain native transparency.

Precision and execution

224 large attention/MLP projections run with FP8 weights and dynamically quantized per-token FP8 activations. Weight scales are per output channel; all scales and smoothing factors are stored in FP32. GEMM accumulates in FP32 (use_fast_accum=False) and emits BF16. Conditioning, input/output projections and normalization retain their original higher precision. The transformer checkpoint occupies 7.261 GB, including scales and unquantized tensors.

Activation smoothing and row quantization use a fused Triton kernel. torch._scaled_mm dispatches to native CUTLASS SM120 FP8 Tensor Core GEMM, verified on an RTX PRO 6000 Blackwell 96 GB. An actual denoising-step trace records 224 FP8 GEMM calls. See kernel evidence and Chrome profiler trace. BF16 is still used for attention, VAE, encoder and the listed sensitive components.

This is a custom Diffusers/PyTorch runtime format, not a drop-in generic Transformers, ComfyUI, vLLM or TensorRT checkpoint. Tested on SM120, Torch 2.14.0 / CUDA 13.0, Transformers 5.17.0 and the pinned Diffusers commit below. Other GPU architectures are unverified. Do not cast the loaded transformer to BF16/FP16; doing so would change FP8 weights and FP32 scales.

Calibration and quality

64 deterministic BF16 trajectories:56 generation cases and 8 edits, 40 denoising steps each. The set spans photographs, portraits, fine textures, illustration, typography, multiple languages, transparency, compositions and aspect ratios, with both 1024 and 2048 resolution. Every quantized projection samples 4 token rows at steps 0, 3, 9, 19, 29, 39: 1536 activation rows per layer. Per-layer activation-aware smoothing and weight clipping are chosen using 384 rows; another 384 rows provide a disjoint diagnostic check. Five smoothing settings and three clipping factors are evaluated with actual FP8 GEMMs. Layers exceeding 5% diagnostic output NRMSE remain BF16. Actual retained outliers: 0.

Calibration prompts and seeds are published in the manifest, and every search result is in calibration_search.json. This is post-training calibration, not QAT or distillation.

Evaluation uses 16 separate generation prompts and 2 separate edits, matched seeds, 40 steps, the same prefix KV-cache setting and the same BF16 encoder/VAE. At 512 px for perceptual comparison, mean LPIPS(AlexNet) is 0.0345 (lower is closer), mean SSIM is 0.9713 (higher is closer). Mean full-resolution final-latent cosine similarity is 0.995627. These measure agreement with the BF16 reference; they do not establish a broad human-preference score or guarantee every prompt. See all paired comparisons and per-image metrics. Rendering may change under a different kernel, precision, dependency version or KV-cache setting.

Full-compute runtime update

The default generate.py now uses decode-only dynamic compilation and kernel fusion, keeping all 40 denoising steps, original calibrated FP8 weights/scales, FP32 GEMM accumulation, and native BF16 attention. No step reuse, distilled adapter, further attention quantization, or reduced resolution is enabled. --eager selects the original runtime.

Same RTX PRO 6000, batch 1, 40 steps, CFG 1, prefix KV cache. CUDA-synchronized end-to-end generation includes encoding and VAE. Warmup/model loading/PNG writes excluded. Original control has 2 measured repeats per size; optimized has 5 at 1024 and 3 at 2048. Raw measurements include first-use warmup and load costs. First-use compilation costs extra time. --warmup reports that cost separately; --prompts-json prompts.json processes a list of prompt strings in one loaded pipeline.

All 18 paired checks were visually reviewed, including primary English/Chinese titles, portraits, transparency and edits. No obvious general visual-quality loss was observed in this finite set. Outputs are not bit-identical: fine textures, decorative typography and some pottery positioning change with GPU reduction rounding. Mean LPIPS versus original FP8=0.014105, mean SSIM=0.985841; worst LPIPS=0.104154 (pottery). Relative to BF16, mean LPIPS=0.035437, compared with 0.034470 for the original FP8. These are fidelity checks, not a guarantee for every prompt. Comparisons · Detailed report.

Updated 30-second real-time image-only video. One completed warmup image is shown at time 0, then each new image appears when actual generation finishes. All waits remain at 1× speed. Frame/request timestamps.

Initial uncompiled latency

RTX PRO 6000 Blackwell 96 GB; batch 1; 40 steps; CFG 1; full GPU residency; prefix KV cache enabled. CUDA-synchronized wall time includes prompt encoding, denoising and VAE decode; excludes model loading and PNG writing. One warmup per resolution is excluded; 1024 has 5 measured repeats and 2048 has 3. No reduced-step adapter or timestep skipping is used.

Raw BF16 measurements · Raw FP8 measurements

Run

Use Linux, Python 3.12 and an NVIDIA Blackwell GPU with a compatible CUDA 13 driver. Install the tested CUDA-enabled PyTorch build and requirements in a new environment. Review the supplied loader before executing it.

python -m venv .venv
source .venv/bin/activate
pip install torch==2.14.0 --index-url https://download.pytorch.org/whl/cu130
pip install -r requirements.txt
python generate.py --prompt "A kingfisher above a forest stream, wildlife photography" --width 1024 --height 1024 --steps 40 --seed 42 --output image.png

Download this repository first with hf download ProCreations/Image-2.1-Calibrated-FP8 --local-dir Image-2.1-Calibrated-FP8, then run the commands from its directory. The loader fetches the unmodified pipeline components from upstream revision b3179ad355be050328e483a9dfdd9e60cd62adfa. To reuse a complete local upstream snapshot, pass --base /path/to/snapshot. For editing, pass --image input.png; for multiple references, pass --image first.png second.png. See the image-editing examples below. Use explicit dimensions and 40 steps to match the benchmark.

Library use: call accelerate_pipeline(pipe) from acceleration.py after load_pipeline(...). The underlying load_pipeline function retains its original eager behavior. The default command uses compilation.

Source model: Qwen/Qwen-Image-2.1@b3179ad355be050328e483a9dfdd9e60cd62adfa. Diffusers: 80c7ed262aeffbeb43ef13ae04baeb9b84515a69.

Real-time video

Watch/download the 30-second demonstration. 1024×1024, 40 steps, image-only, no audio or overlays. It starts with one completed warmup image; each subsequent change was captured live when another image finished. All intervening waits remain at 1× speed. Capture timestamps record request start, generation completion, display updates and every video frame.

Image editing and multiple reference images

--image PATH supplies a local input/reference image to guide an edit. Use --prompt to describe the requested changes; --output is the newly generated result. Omitting --image selects text-to-image generation.

Single-image edit:

python generate.py --image input.png --prompt "Change the dog's fur to silver gray, keeping its pose and the background." --width 1024 --height 1024 --steps 40 --output edited.png

Pass several paths after --image to combine multiple references in one generation:

python generate.py --image rgba/corgi.png rgba/dragon.png --prompt "Place the corgi from image 1 next to the mint-green dragon from image 2. Keep both characters recognizable, full body, on a transparent background." --transparent --width 1024 --height 1024 --steps 40 --output together.png

Repeating the flag is equivalent:

python generate.py --image rgba/corgi.png --image rgba/dragon.png --prompt "Place the corgi from image 1 next to the dragon from image 2." --width 1024 --height 1024 --steps 40 --output together.png

References keep command-line order: the first is image 1, the second image 2, and so on. Quote paths containing spaces. All supplied images jointly condition each output; they are not processed as independent edits. With --prompts-json, the same ordered references apply to every prompt. To use different references for each edit, run separate commands. Output dimensions remain controlled by --width and --height (both default to 2048); reference images are resized by the pinned upstream pipeline while preserving their aspect ratios.

The CLI checks that images are readable before loading the model and reports input_image_count. The existing single-image syntax is preserved. Multiple references increase conditioning work and memory, so the text-only benchmark does not describe multi-image editing speed. --cache-prompts bypasses all image-conditioned requests. Model weights, quantization and denoising are unchanged.

Validated on the RTX PRO 6000 with two native RGBA references and all 40 steps. Both multi-image spellings produced identical RGBA pixels, and a single-image edit matched the previous CLI pixel for pixel. Validation details and sample.

Reproduce

calibrate.py, quantize.py, evaluate.py, metrics.py and video.py are the original experiment scripts; their workstation paths are explicit and should be changed for a different installation. Run BF16 evaluation before FP8 evaluation because paired edits use the same BF16 reference input. Raw sampled activation caches are generated locally by calibration and are not required for inference. The published checkpoint uses safetensors.

License and modifications

The original model is under the Qwen Research License, allowing non-commercial research/evaluation subject to its terms. Preserve Notice and the license when redistributing. Modified weights include a prominent modification notice in safetensors metadata and the quantization configuration. This is an independent ProCreations quantization, not an official Qwen release.