ideogram-v4-fast-nunchaku-lite-nvfp4

Ideogram v4 Fast — nunchaku-lite NVFP4 (calibrated SVDQuant)

Calibrated SVDQuant (NVFP4 (fp4, group size 16) weights with int4 W4A16 on AdaLN) quantization of Ideogram v4 fast, packaged as a ready-to-run Diffusers pipeline. ~16 GB vs the bf16 original.

Derived from the gated fal/ideogram-v4-fast; the text encoder + VAE components come from ideogram-ai/ideogram-4-nf4-diffusers.

Samples — bf16 vs int4 vs nvfp4 (this repo is NVFP4)

Same "GOOD MORNING" caption/prompt (see below), 1024×1024, 20 steps:

The main design renders well at both int4 and nvfp4; only small text differs from bf16.

Load & generate

import json, torch
from diffusers import DiffusionPipeline

pipe = DiffusionPipeline.from_pretrained(
    "lite-infer/ideogram-v4-fast-nunchaku-lite-nvfp4",
    torch_dtype=torch.bfloat16,
).to("cuda")

# Ideogram v4 expects a STRUCTURED JSON caption (a plain string will not render):
prompt = json.dumps({
    "high_level_description": "A bold minimalist typographic poster with a friendly sunrise motif",
    "compositional_deconstruction": {
        "background": "Warm white textured paper with a soft peach gradient at the bottom",
        "elements": [
            {"type": "text", "text": "GOOD MORNING", "desc": "large bold rounded sans-serif title in deep navy"},
            {"type": "illustration", "desc": "a simple flat sun with rays rising behind the text in warm orange"},
        ],
    },
})

image = pipe(
    prompt, height=1024, width=1024, num_inference_steps=20,
    guidance_scale=1.0, guidance_schedule=None, mu=0.0, std=1.75,
    generator=torch.Generator("cuda").manual_seed(42),
).images[0]
image.save("out.png")

guidance_scale=1.0 + guidance_schedule=None are required (the pipeline's zero unconditional branch means the default guidance schedule over-amplifies and produces garbage).

Requirements

The quantized weights run through the nunchaku fp4/int4 kernels:

• pip install "kernels==0.16" and PyTorch 2.11+cu128
• a Diffusers build that includes Ideogram4Pipeline
• set DIFFUSERS_TRUST_REMOTE_KERNELS=true (kernels are fetched from rootonchair/nunchaku-lite-kernels)

Requires a Blackwell (sm_120) GPU for the fp4 kernels.

Method

Calibrated SVDQuant produced with diffuse-compressor (examples/text_to_image/quantize_ideogram_v4.py): NVFP4 (fp4, group size 16) weights with int4 W4A16 on AdaLN, a rank-32 low-rank branch (search mode), grid-search projection smoothing, AWQ W4A16 on the AdaLN modulation projections, no GPTQ; calibrated on Ideogram JSON-caption prompts replayed through chained per-block scopes.

Notes

• Ideogram v4 has a model-level safety filter that occasionally returns a blank "Image blocked by safety filter" card for otherwise benign prompts — re-roll the seed if that happens.
• Small text rendering is imperfect and slightly worse than the bf16 model.