Qwen-Image-2.1-PE-I2I-FP8

Qwen-Image-2.1-PE-I2I-FP8

Qwen-Image-2.1-PE-I2I-FP8 is an FP8 dynamic-quantized build of Qwen/Qwen-Image-2.1-PE-I2I, the image editing prompt rewriting model for Qwen-Image-2.1. The base model is a fine-tuned Qwen3.5-VL 9B that takes a vague editing instruction plus one or more input images and produces a precise, actionable prompt for the downstream image editing model. This checkpoint was compressed with llm-compressor using the FP8_DYNAMIC scheme, stored in the compressed-tensors format, and is intended to be served with vLLM. The Linear layers of the language model are quantized to FP8 weights with dynamic per-token FP8 activations, which lowers weight memory and improves serving throughput relative to the BF16 original. The vision tower, lm_head, embedding layers, and linear attention layers are left in their original precision. No calibration data is required, because activation scales are computed at runtime.

[!NOTE] System Prompt — https://huggingface.co/Qwen/Qwen-Image-2.1-PE-I2I/blob/main/system_prompt.txt

Quantization Details

Recipe

default_stage:
  default_modifiers:
    QuantizationModifier:
      targets: [Linear]
      ignore: ['re:.*lm_head', 're:.*embed_tokens$', 're:.*visual.*', 're:.*model.visual.*',
        're:.*linear_attn.*']
      scheme: FP8_DYNAMIC
      bypass_divisibility_checks: false
      requires_calibration_data: false

Reproduction

import torch
from transformers import AutoModelForImageTextToText, AutoProcessor
from llmcompressor import oneshot

model_id = "Qwen/Qwen-Image-2.1-PE-I2I"
save_dir = "Qwen-Image-2.1-PE-I2I-FP8"

model = AutoModelForImageTextToText.from_pretrained(
    model_id, dtype=torch.bfloat16, device_map="auto"
)
processor = AutoProcessor.from_pretrained(model_id)

# recipe.yaml is the recipe shown above
oneshot(model=model, recipe="recipe.yaml")

model.save_pretrained(save_dir, save_compressed=True)
processor.save_pretrained(save_dir)

Quick Start

Installation

pip install vllm openai pillow huggingface_hub

Use a recent vLLM release with support for Qwen3.5-VL and compressed-tensors FP8 checkpoints. Native FP8 compute requires a GPU with compute capability 8.9 or higher (Ada Lovelace, Hopper, Blackwell). On older GPUs vLLM falls back to weight-only FP8 kernels.

Serve with vLLM

vllm serve prithivMLmods/Qwen-Image-2.1-PE-I2I-FP8 \
    --max-model-len 32768

For multi-image inputs, raise the per-prompt image limit with --limit-mm-per-prompt. Do not enable a reasoning parser, so the thinking block is returned in the response content and can be split from the JSON answer as shown below.

Query the Server

import base64
import json

import huggingface_hub
from openai import OpenAI

model_id = "prithivMLmods/Qwen-Image-2.1-PE-I2I-FP8"
client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

# System prompt shipped with the base model
sys_prompt_path = huggingface_hub.hf_hub_download(
    "Qwen/Qwen-Image-2.1-PE-I2I", "system_prompt.txt"
)
system_prompt = open(sys_prompt_path).read().strip()


def to_data_url(path):
    with open(path, "rb") as f:
        return "data:image/png;base64," + base64.b64encode(f.read()).decode()


response = client.chat.completions.create(
    model=model_id,
    messages=[
        {"role": "system", "content": system_prompt},
        {"role": "user", "content": [
            {"type": "image_url", "image_url": {"url": to_data_url("input.png")}},
            {"type": "text", "text": "make the sky sunset"},
        ]},
    ],
    max_tokens=24000,
    temperature=1.0,
    top_p=0.95,
    seed=42,
    extra_body={"top_k": 20},
)

gen = response.choices[0].message.content

# Split thinking from the answer
thinking, _, answer = gen.partition("</think>")
result = json.loads(answer.strip())
print(result)
# {"rewritten_prompt": "...", "wh_ratio": "", "ratio_follow": "<image1>"}

Multi-Image Editing

Multiple input images are referenced as <image1>, <image2>, and so on. Add one image_url entry per image before the text part:

messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": [
        {"type": "image_url", "image_url": {"url": to_data_url("portrait.png")}},
        {"type": "image_url", "image_url": {"url": to_data_url("scene.png")}},
        {"type": "text", "text": "Place <image1>'s subject into <image2>'s scene"},
    ]},
]

Integration with Diffusers

import torch
from PIL import Image
from diffusers import QwenImage21Pipeline

# `result` from the vLLM call above
prompt = result["rewritten_prompt"]
input_image = Image.open("input.png").convert("RGB")

pipe = QwenImage21Pipeline.from_pretrained(
    "Qwen/Qwen-Image-2.1", torch_dtype=torch.bfloat16
).to("cuda")

image = pipe(
    prompt=prompt,
    image=input_image,
    num_inference_steps=40,
    generator=torch.Generator("cuda").manual_seed(42),
).images[0]

image.save("rewritten_edit.png")

Output Format

The model emits a JSON object after a <think> reasoning block:

{
  "rewritten_prompt": "<precise editing instruction>",
  "wh_ratio": "",
  "ratio_follow": "<image1>"
}

wh_ratio and ratio_follow are mutually exclusive, and exactly one carries a value.

Notes

• FP8 quantization introduces small numerical differences from the BF16 original, so rewritten prompts may differ slightly in wording. Validate output quality on your own editing workloads before replacing the base model.
• Always parse the answer with a JSON loader and handle malformed output, since sampling at temperature=1.0 can occasionally produce invalid JSON.
• The vision tower is kept in its original precision, so image encoding memory and latency are unchanged from the base model.

License

This model inherits the license of the base model and is distributed under the Qwen Research License Agreement.

Acknowledgements

Base model by the Qwen team: Qwen/Qwen-Image-2.1-PE-I2I. Quantization with llm-compressor. Serving with vLLM. See the Qwen-Image-2.1 GitHub repo and blog for details on the full pipeline.