Ming-Image-0.1-Design-ROCm-INT8

Ming-Image-0.1-Design — ROCm build (AMD Strix Halo, gfx1151) · INT8 MLLM · paired with Ling-3.0-flash-VL

inclusionAI/Ming-Image-0.1-Design (text-to-image for UI, posters and infographics, RGBA output) made to run on AMD ROCm, with the routed experts of its 17B-parameter MoE language model stored as weight-only INT8, and wired to the prompt enhancer its model card names: Ling-3.0-flash-VL (served from our ROCmFP4 build).

Everything below was measured on one AMD Ryzen AI Max+ 395 (Radeon 8060S, gfx1151) — see Reproduction. Nothing here was run on CUDA.

Results (1024 × 1024, 12 steps, cfg 1.0, seed 42, four prompts)

• Contiguous attention is the big one: 1.78× faster end to end, and the images are byte-identical to the upstream code path (4/4 prompts plus the --release run, compared with cmp).
• † INT8 runs the same BF16 diffusion transformer, so it does not change the step time: its 227.7 s mean vs 253.6 s for BF16 with the same attention reflects when the runs happened (later, with Ling resident), not the quantization. What INT8 does cost is the conditioning pass, where the weights are dequantized on the fly: 2.4–4.7 s vs 1.1–2.0 s in BF16. INT8 exists for memory: with --release-mllm-after-conditioning Ming needs 35.8 GiB at its peak, which fits next to the resident Ling-3.0-flash-VL (67.5 GiB) on a 128 GiB box — BF16 with the upstream code needed 74.5 GiB and did not. Loading takes 30–100 s; the slow end is a cold page cache (first load after the files were written or after Ling was restarted).
• Fast attention (--attention-bf16-reduction) is opt-in: 5.21× vs upstream, at the fidelity cost shown below.
• The very first image on a fresh box is slower once: the reference cabin took 608.4 s the first time and 339.9 s warm (MIOpen tunes the VAE's 3-D convolutions and caches the result in ~/.cache/miopen).

Fidelity against the BF16 original

Inference on this box is deterministic: the same prompt and seed produced a byte-identical PNG twice, in BF16 and in INT8, so every difference below is caused by the quantization (or by the fast-attention option), not by run-to-run noise.

cos / rel L2 compare the conditioning tensors the diffusion transformer receives (the MLLM output — the only part that is quantized). SSIM is windowed 7×7 on luminance; PSNR over RGB.

Run it

hf download kingjones777/Ming-Image-0.1-Design-ROCm-INT8 --local-dir Ming-Image-0.1-Design-ROCm-INT8
cd Ming-Image-0.1-Design-ROCm-INT8
# caption -> Ling-3.0-flash-VL rewrites it into the layered JSON prompt -> Ming renders it (one image per process)
bash code/generate_paired.sh --model . --base-url http://127.0.0.1:8090/v1 \
    "A minimalist concert poster for a jazz night called \"Blue Hour\"" --resolution 1024 --output-dir out \
    -- --device-map none --release-mllm-after-conditioning        # add --attention-bf16-reduction for fast mode

PYTHON=/path/to/python selects the interpreter (it needs a ROCm build of PyTorch and code/requirements-rocm.txt); --pe-model names the chat model your endpoint serves. This is the command shape our end-to-end run used (see Samples); 1024² is the resolution we measured — infer.py's own default is 2048², which we did not run.

generate_paired.sh calls pe_ling.py (standard library only) against an OpenAI-compatible endpoint (default http://127.0.0.1:8090/v1, our Ling-3.0-flash-VL llama-server seat), validates the rewrite against the schema the upstream system prompt demands (one retry with the errors), then runs infer.py. You can also pass your own JSON prompt straight to infer.py --prompt prompt.json.

The code is in code/: upstream inclusionAI/Ming-Image at 62c6072 plus the changes below (code/rocm.patch).

What changed for ROCm, and why

1. transformer_engine removed. The vision tower imported NVIDIA Transformer Engine (CUDA-only) for a single te.RMSNorm; it is now a plain RMSNorm with the same parameter name (weight), so all 65 vision norm tensors in the checkpoint load unchanged. A dead import in modeling_bailing_moe_v2.py went too.
2. --attn-implementation eager now reaches the towers. BailingMM2Config declared no sub_configs, so transformers never copied the chosen attention implementation into the vision and language configs; their flash_attention_2 defaults raised ImportError at model construction on any box without flash-attn. Declaring sub_configs fixes it. (Upstream's --validate-only cannot catch this — it never builds the model.)
3. Contiguous attention inputs. The diffusion transformer handed PyTorch's attention permuted views of [B, L, H, D] tensors. On gfx1151 the only working SDPA kernel is the math one, and with those strides its fp32 GEMMs fall onto an 8×8×8 macro-tile: 644 ms per attention call at the cabin prompt's length (L = 5,759) versus 363 ms for the same tensors made contiguous, bit-identical output. The profiler put SDPA-math at 87.6% of all GPU time before the fix. diffusion/transformer.py now passes contiguous tensors whenever diffusers' default native backend is active.
4. INT8 MLLM — routed experts only. Weight-only, per-output-channel symmetric INT8 (fp32 scales) for the 256 routed experts of the language model's 19 MoE layers — 14,592 Linears, which hold almost all of its weights. Everything that runs on every token stays byte-identical BF16: attention (query_key_value, dense), the shared experts, layer 0's dense MLP, the routers (gate, image_gate, audio_gate), embeddings, lm_head, norms and the vision tower (691 tensors). mllm/ goes from 34.00 GB to 18.76 GB. Relative weight error ‖W − Ŵ‖/‖W‖: mean 0.00833, p99 0.01035, max 0.01412. quant/quantize_stream.py writes it shard by shard without building the model (6.7 min; peak RSS 17.84 GiB, sampled on an earlier run of the same tool); quant/load_int8.py builds the model on the meta device and loads the INT8 shards straight onto the GPU, so BF16 weights for the quantized layers never exist in memory. The scales stay fp32 through .to(bfloat16). Measured alternative: quantizing the always-on Linears too saves another 0.30 GB but raised the mean conditioning error (rel L2, VLM tokens) from 0.090 to 0.108 and changed the cabin prompt's surround from scenery to white (SSIM 0.699); mean SSIM over the four prompts 0.877 vs 0.924.
5. Connector stored as bf16. It shipped as float32 (6.17 GB) but infer.py always loads it as bf16; storing it pre-rounded halves the download and every tensor equals fp32.to(bfloat16) exactly (all 338 checked).
6. --release-mllm-after-conditioning (opt-in, one image per process): the language model, vision tower and connector are only needed for the ~1–2 s conditioning pass, so they are freed before the 12 diffusion steps. PyTorch peak allocated on the cabin prompt (BF16): 55.6 → 47.2 GiB, byte-identical image.
7. --attention-bf16-reduction (opt-in): lets the math SDPA kernel stay in bf16 instead of upcasting to fp32 — 363 → 115 ms per call, attention error vs an fp32 reference 1.658e-03 → 5.032e-03 (rel L2, random inputs). Image-level cost is in the fidelity table.

Things that do not work on gfx1151 (measured, so you don't have to)

• Only the math SDPA kernel runs. AOTriton's efficient and flash kernels are unavailable by default (UNAVAILABLE: No available kernel. Aborting execution.).
• Do not set TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1, even though PyTorch's warning suggests it: the shipped amd-gfx11xx AOTriton kernels then get selected and fail (UNAVAILABLE: HIP error: invalid argument), and a full generation crashes with the same HIP error: invalid argument.
• The first image on a fresh box is slower once. The same cabin prompt took 608.4 s the first time and 339.9 s warm; MIOpen tunes the VAE's 3-D convolutions on first use and its cache (~/.cache/miopen) grew during that first run.
• Inference is deterministic here, BF16 and INT8: same prompt + seed → byte-identical PNG (both checked), which is what makes the fidelity numbers above exact rather than statistical.

Pairing with Ling-3.0-flash-VL

Ming-Image's text-to-image quality depends on a structured, Figma-like JSON prompt (canvas settings + positioned layers with colours). Upstream ships the rewriter system prompt (assets/t2i_rewriter_system_prompt.txt) and names Ling-3.0-flash-VL as the model to run it. pe_ling.py sends that prompt verbatim plus your caption to any OpenAI-compatible endpoint, extracts the JSON, validates it, and retries once with the validation errors if needed.

Measured against our Ling-3.0-flash-VL seat (llama-server, ling-3.0-flash-vl-mtp-halo-STRIX_LEAN, same box) — all three valid on the first attempt:

The exact captions and the JSON Ling returned are in samples/.

Samples

INT8 build, --release-mllm-after-conditioning, 1024², 12 steps, cfg 1.0, seed 42 — exactly the runs in the tables above. Full-resolution RGBA PNGs and the JSON prompts are in samples/.

cabin upstream — upstream's own structured prompt (assets/t2i_four_seasons_cabin_prompt.json), no rewrite · prompt JSON

ui banking — caption → Ling-3.0-flash-VL: A mobile banking app home screen: a balance card at the top, a recent transactions list, quick-action buttons for send, pay and top up, and a bottom navigation bar. Clean modern fintech style. · prompt JSON

poster jazz — caption → Ling-3.0-flash-VL: A minimalist concert poster for a jazz night called "Blue Hour" on Friday, October 3, 8 PM at The Lantern Room, with a saxophone silhouette over a deep blue gradient. · prompt JSON

info water — caption → Ling-3.0-flash-VL: An infographic that explains the water cycle in four labeled stages - evaporation, condensation, precipitation, collection - with arrows and simple flat icons. · prompt JSON

End to end on the box

ming-paired (our box's wrapper around generate_paired.sh, not part of this repo) with Ling-3.0-flash-VL and two other model servers running: it paused 2 of them for the render and restarted them afterwards (halo-bonsai8b, halo-bonsai). Ling rewrote the caption in 48.3 s (9 layers); the whole command took 323 s.

A landing page hero section for a coffee subscription service called Daily Grind: the headline Fresh beans every Monday, a short tagline, a Start your subscription button, and a photo of latte art on the right. Warm earthy palette. · prompt JSON

Files

Reproduction

upstream : inclusionAI/Ming-Image-0.1-Design  revision 1cd7fac3b0dcb54196fe2cd12b80da09edf8fcf4
code     : inclusionAI/Ming-Image @ 62c6072e1ff15af83f7c4963a0a1954c1424e80e + code/rocm.patch  (branch rocm-halo @ f986f7a (upstream inclusionAI/Ming-Image 62c6072 + 3 commits); rocm.patch sha256 03fe16f8ed566bf99ec654caa687c7e659f489df6d077cfb7fc7aee134ebc777)
python   : 3.13.5 · torch 2.10.0 (HIP 7.13.99004) · transformers 4.57.1 · diffusers 0.36.0 · accelerate 1.13.0 · safetensors 0.8.0
box      : amd-halo · AMD RYZEN AI MAX+ 395 w/ Radeon 8060S · 125 GiB RAM visible, 128 GiB GTT · ROCm 7.13.0 · kernel 6.18.35+rex+2-amd64
power    : platform_profile=balanced · governor=powersave · GPU 83–107 W at 100% busy (step probe, 0.5 s samples)
env      : nothing set; in particular TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL is NOT set (it crashes gfx1151)
date     : 2026-09-22 to 2026-09-23 (runs crossed midnight, America/Chicago)

Build the INT8 package from the upstream download:

python code/quant/quantize_stream.py <upstream>/mllm <package>/mllm \
    --exclude '\.attention\.|\.shared_experts\.|^model\.model\.layers\.0\.mlp\.'   # routed experts only
python code/tools/convert_connector.py <upstream>/connector <package>/connector # fp32 -> bf16, proven exact
# transformer/ vae/ mlp/ scheduler/ LICENSE are the upstream files, unchanged
python code/tools/verify_package.py <upstream> <package>                        # the checks behind this card

Measure (one process per image, the pairing configuration):

cd code && PYTHONPATH=. python tools/ming_bench.py --prompts <prompt.json> --out <dir> -- \
    --model <package> --task text-to-image --resolution 1024 --device-map none \
    --attn-implementation eager --release-mllm-after-conditioning        # add --attention-bf16-reduction for fast
python tools/fidelity_compare.py <bf16-dir> <int8-dir> --json fidelity.json

BF16 references ran with Ling stopped (the box cannot hold both); every INT8 run had Ling-3.0-flash-VL resident. GPU memory is the amdgpu mem_info_gtt_used peak sampled every second, minus a baseline taken after GTT settled.

License

MIT, same as the original. Model weights, architecture and inference code © 2026 inclusionAI (LICENSE). The INT8 quantization, the ROCm changes and the pairing scripts are ours; they are listed in this card and in code/rocm.patch.