Z-Image-Turbo-FP8-attn-out-only

Z-Image-Turbo FP8 — attn-out-only

Only the attention output projection (to_out) is FP8; everything else stays in BF16.

★ The overall best practical choice in our benchmark is online_fp8 (5.937 s/image, -24.2% vs BF16, peak GPU 13.34 GB — 38.4% less than BF16). It runs straight from the upstream Tongyi-MAI/Z-Image-Turbo checkpoint with --quantization-config '{"method":"fp8"}' — no separate download. The pre-quantized variants in this family are useful when you want deterministic static scales (calibrated offline, frozen on disk) or when online quantization is not desired.

This variant (attn-out-only) → mean latency 7.623 s/image (-2.7% vs BF16), peak GPU 21.18 GB (2.2% less than BF16's 21.66 GB), image quality PSNR 26.93 dB / SSIM 0.9027 vs the BF16 reference at identical (prompt, seed).

This is one of 7 FP8 quantization policies of Tongyi-MAI/Z-Image-Turbo prepared for vLLM-Omni serving. Each variant differs only in which transformer submodules are quantized to FP8 (E4M3 with per-tensor static scales); everything else stays in BF16.

In the tables below, ★ marks the overall best choice (online_fp8) and ← marks the row that corresponds to this repo.

Quick start (vLLM-Omni)

docker run --rm --gpus all --ipc=host \
  -p 18002:8002 \
  -e HF_HOME=/hf_cache \
  -e DIFFUSION_ATTENTION_BACKEND=TORCH_SDPA \
  -e VLLM_ATTENTION_BACKEND=FLASH_ATTN \
  -v /your/hf/cache:/hf_cache \
  your-vllm-omni:latest \
    vllm-omni serve bahadirakdemir/Z-Image-Turbo-FP8-attn-out-only \
    --omni --host 0.0.0.0 --port 8002 \
    --gpu-memory-utilization 0.90 \
    --tensor-parallel-size 1 \
    --quantization-config '{"method":"modelopt","quant_method":"FP8","is_checkpoint_fp8_serialized":true,"kv_cache_quant_method":null,"exclude_modules":["model*","lm_head*","*feed_forward*","*to_qkv*"]}'

Then call the OpenAI-style image API:

curl -s http://127.0.0.1:18002/v1/images/generations \
  -H "Content-Type: application/json" \
  -d '{
    "model": "bahadirakdemir/Z-Image-Turbo-FP8-attn-out-only",
    "prompt": "a cinematic photograph of an old fisherman standing on a foggy pier at sunrise",
    "size": "1024x512",
    "num_inference_steps": 8,
    "guidance_scale": 0.0,
    "n": 1,
    "response_format": "b64_json"
  }'

Benchmark — all 7 FP8 policies at 8 steps, 1024×512 on NVIDIA GB10

Same hardware, same (prompt, seed) tuples, same Docker image, fresh container per variant.

Pass A — DIFFUSION_ATTENTION_BACKEND=SAGE_ATTN

PSNR / SSIM are computed against the BF16 reference image at the same prompt and seed. Higher is better; BF16 is ref (it's the reference).

Pass B — top variants at DIFFUSION_ATTENTION_BACKEND=TORCH_SDPA

On NVIDIA GB10 (compute capability SM 12.1), FLASH_ATTN is platform-gated and silently falls back to TORCH_SDPA. SageAttention has a small but consistent overhead vs TORCH_SDPA on Blackwell, so the production recommendation is TORCH_SDPA.

Image comparisons

Same 3 prompts rendered across every variant with identical seeds. Click for full resolution.

Prompt 0 — "a cinematic photograph of an old fisherman standing on a foggy pier at sunrise"

Prompt 1 — "a detailed watercolor painting of a mountain village beside a clear blue lake"

Prompt 2 — "a futuristic city street at night with neon reflections on wet pavement"

Benchmark parameters

All 7 repos in this family

• bahadirakdemir/Z-Image-Turbo-FP8-full
• bahadirakdemir/Z-Image-Turbo-FP8-ffn-only
• bahadirakdemir/Z-Image-Turbo-FP8-ffn-attn-out
• bahadirakdemir/Z-Image-Turbo-FP8-ffn-attn-qkv
• bahadirakdemir/Z-Image-Turbo-FP8-attn-only
• bahadirakdemir/Z-Image-Turbo-FP8-attn-out-only ← you are here
• bahadirakdemir/Z-Image-Turbo-FP8-attn-qkv-only

License & attribution

• Apache 2.0, inherited from upstream Tongyi-MAI/Z-Image-Turbo.
• Quantization: NVIDIA ModelOpt FP8 PTQ with 8 calibration prompts × 9 denoise steps each.
• Serving: vLLM-Omni.
• Benchmarked on NVIDIA GB10 (Grace Blackwell, SM 12.1), 2026-05-17.