Qwen-Image-2.1-PE-T2I-FP8

Qwen-Image-2.1-PE-T2I-FP8

Qwen-Image-2.1-PE-T2I-FP8 is an FP8 dynamic-quantized build of Qwen/Qwen-Image-2.1-PE-T2I, the text-to-image prompt rewriting model for Qwen-Image-2.1. The base model is a fine-tuned Qwen3.5-VL 9B that turns a brief image request in any language into a detailed English prompt plus a recommended aspect ratio. This checkpoint was compressed with llm-compressor using the FP8_DYNAMIC scheme, stored in the compressed-tensors format, and is intended to be served with vLLM. The Linear layers are quantized to FP8 weights with dynamic per-token FP8 activations, which lowers weight memory and improves serving throughput relative to the BF16 original. The lm_head, embedding layers, vision modules, and linear attention layers are excluded and kept in their original precision. No calibration data is required, because activation scales are computed at runtime.

[!NOTE] System Prompt — https://huggingface.co/Qwen/Qwen-Image-2.1-PE-T2I/blob/main/system_prompt.txt

Quantization Details

Recipe

default_stage:
  default_modifiers:
    QuantizationModifier:
      targets: [Linear]
      ignore: ['re:.*lm_head', 're:.*embed_tokens$', 're:.*visual.*', 're:.*model.visual.*',
        're:.*linear_attn.*']
      scheme: FP8_DYNAMIC
      bypass_divisibility_checks: false
      requires_calibration_data: false

Reproduction

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from llmcompressor import oneshot

model_id = "Qwen/Qwen-Image-2.1-PE-T2I"
save_dir = "Qwen-Image-2.1-PE-T2I-FP8"

model = AutoModelForCausalLM.from_pretrained(
    model_id, dtype=torch.bfloat16, device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_id)

# recipe.yaml is the recipe shown above
oneshot(model=model, recipe="recipe.yaml")

model.save_pretrained(save_dir, save_compressed=True)
tokenizer.save_pretrained(save_dir)

Quick Start

Installation

pip install vllm openai huggingface_hub

Use a recent vLLM release with support for Qwen3.5-VL and compressed-tensors FP8 checkpoints. Native FP8 compute requires a GPU with compute capability 8.9 or higher (Ada Lovelace, Hopper, Blackwell). On older GPUs vLLM falls back to weight-only FP8 kernels.

Serve with vLLM

vllm serve prithivMLmods/Qwen-Image-2.1-PE-T2I-FP8 \
    --max-model-len 32768

Do not enable a reasoning parser, so the thinking block is returned in the response content and can be split from the JSON answer as shown below.

Query the Server

import json

import huggingface_hub
from openai import OpenAI

model_id = "prithivMLmods/Qwen-Image-2.1-PE-T2I-FP8"
client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

# System prompt shipped with the base model
sys_prompt_path = huggingface_hub.hf_hub_download(
    "Qwen/Qwen-Image-2.1-PE-T2I", "system_prompt.txt"
)
system_prompt = open(sys_prompt_path).read().strip()

user_prompt = "一只在雨中弹吉他的柯基"

response = client.chat.completions.create(
    model=model_id,
    messages=[
        {"role": "system", "content": system_prompt},
        {"role": "user", "content": user_prompt},
    ],
    max_tokens=16256,
    temperature=1.0,
    top_p=0.95,
    seed=42,
    extra_body={"top_k": 20},
)

gen = response.choices[0].message.content

# Split thinking from the answer
thinking, _, answer = gen.partition("</think>")
result = json.loads(answer.strip())
print(result)
# {"rewritten_prompt": "<long detailed English prompt>", "wh_ratio": "16:9"}

Integration with Diffusers

import torch
from diffusers import QwenImage21Pipeline

WH_RATIO_TO_SIZE = {
    "1:1": (2048, 2048), "4:3": (2400, 1792), "3:4": (1792, 2400),
    "3:2": (2528, 1696), "2:3": (1696, 2528), "16:9": (2752, 1536),
    "9:16": (1536, 2752),
}

# `result` from the vLLM call above
prompt = result["rewritten_prompt"]
width, height = WH_RATIO_TO_SIZE.get(result["wh_ratio"], (2048, 2048))

pipe = QwenImage21Pipeline.from_pretrained(
    "Qwen/Qwen-Image-2.1", torch_dtype=torch.bfloat16
).to("cuda")

image = pipe(
    prompt=prompt,
    width=width, height=height,
    num_inference_steps=40,
    generator=torch.Generator("cuda").manual_seed(42),
).images[0]

image.save("rewritten_t2i.png")

Output Format

The model emits a JSON object after a <think> reasoning block:

{
  "rewritten_prompt": "<long detailed English prompt describing the finished image>",
  "wh_ratio": "16:9"
}

The aspect ratios used in the Diffusers example map to the following output sizes:

Notes

• FP8 quantization introduces small numerical differences from the BF16 original, so rewritten prompts may differ slightly in wording. Validate output quality on your own generation workloads before replacing the base model.
• Always parse the answer with a JSON loader and handle malformed output, since sampling at temperature=1.0 can occasionally produce invalid JSON.
• Input requests can be written in any language, and the rewritten prompt is always English.

License

This model inherits the license of the base model and is distributed under the Qwen Research License Agreement.

Acknowledgements

Base model by the Qwen team: Qwen/Qwen-Image-2.1-PE-T2I. Quantization with llm-compressor. Serving with vLLM. See the Qwen-Image-2.1 GitHub repo and blog for details on the full pipeline.