Ideogram-4-IQ4_M

Ideogram 4 — IQ4_M for mlx-serve

Calibrated 4-bit conversion of ideogram-ai/ideogram-4-fp8, built for mlx-serve, the native Zig MLX server for Apple Silicon.

14.5 GB on disk, 4.71 bits per weight averaged over all 25.2B quantized parameters — and about 10 GB resident for a CFG-free render, because Ideogram's unconditional branch is a whole second transformer that is only loaded when it is actually used (see Residency below).

The +0.5 over each nominal width is the affine group overhead: an f16 scale and an f16 bias per 64 weights.

What "IQ4" means here

Every quantized weight was fitted with an importance matrix — the per-input-channel mean-squared activation, E[x^2] — rather than plain round-to-nearest. An error dW_j in input channel j contributes dW_j^2 * E[x_j^2] to the output, so weighting the least-squares fit by that term spends precision where the model actually puts signal. MLX's own min/max solution is candidate 0 of the search, so a calibrated weight is never worse than the RTN one by that objective; on a representative tensor the weighted reconstruction error came out 88% lower.

It is not llama.cpp's IQ4_NL: the grid here is uniform affine, exactly what mlx_quantized_matmul reads. What is borrowed is the importance matrix and the bits-per-weight naming, not the non-linear codebook.

How the statistics were collected. A text encoder can be calibrated by pushing text through a reference implementation; a diffusion transformer cannot, because its activations depend on the latents and the timestep. So they were collected from real generations — 7 renders across three canvases (512², 896×512, 512×896) at both guidance 1.0 and 7.0, covering photographic, illustrated and typographic captions with Latin, CJK and Cyrillic sign text — giving 673 projections: 252 in the encoder, 211 in the conditional DiT, 210 in the unconditional one. (The asymmetry is real: llm_cond_proj never runs on the unconditional branch, which takes no text conditioning.)

Two honest caveats. The statistics were gathered through an already-quantized pack, since that is the only Ideogram 4 that runs on Apple Silicon — an activation second moment is far more stable to weight noise than the weights are, but it is an approximation. And two weights fall back to round-to-nearest because nothing observes them: the encoder's embed_tokens (a gather table, not a projection) and the unconditional branch's unused llm_cond_proj.

Against the pack this replaces

Same prompt, same seed, same 20 steps — a hand-painted café sign, because text is what this model is for and the first thing quantization takes away.

At 1024² the older pack smears the lettering — a chewed accent, a stray mark before the first character, mushy strokes — while this one renders CAFÉ MOKO cleanly with its painted drop shadow intact.

Two things changed between those images, and honesty requires naming both: the DiT bulk went from 3-bit to 4-bit, and the fit went from round-to-nearest to imatrix-calibrated. This pair does not separate them, and my guess is the extra bit does most of the visible repair. What is isolated — same width, only the fit changing — is the reconstruction-error measurement: 88% lower weighted error on a representative tensor. Attributing the picture to calibration alone would be reading one sample as a trend.

Worth knowing: at 640² the two are comparable (previous, IQ4_M). Small canvases flatter a low-bit pack; the failure shows up at the size you would actually generate at.

Residency

Ideogram 4 uses asymmetric CFG: the negative branch is its own 9.3B checkpoint, not the same weights with an empty prompt. At guidance_scale 1.0 the blend pos·gw + neg·(1−gw) multiplies that branch by exactly zero, so mlx-serve skips the forward and never loads the checkpoint:

The unconditional half is a deferred stage — brought in on the first guided render, and priced against free memory at that moment rather than at load, so a machine that cannot hold both refuses by name instead of dying mid-denoise.

Serving

mlx-serve --model justintime47/Ideogram-4-IQ4_M --serve

curl localhost:8080/v1/images/generations -H 'Content-Type: application/json' -d '{
  "model": "justintime47/Ideogram-4-IQ4_M",
  "prompt": "a hand-painted café sign above a green doorway, 35mm film photograph",
  "size": "1024x1024", "steps": 20
}'

Prompting. Ideogram 4 was trained exclusively on structured JSON captions — bounding boxes, palettes, per-element descriptions — so a bare sentence is out of distribution rather than merely weaker. mlx-serve rewrites one for you with any chat model it already has loaded (magic_prompt, on by default), decoding under a JSON-schema grammar so the caption cannot come back malformed. Write the caption yourself and it is passed through untouched — that is the documented way to control layout and colour by hand.

Turbo. Drop ostris' TurboTime adapter into the model folder as turbo_lora.safetensors and send "turbo": true: 8 steps instead of 20, no CFG, and the second checkpoint stays unloaded.

Conversion

python tests/convert_ideogram4.py \
    --src ideogram-ai/ideogram-4-fp8 \
    --out Ideogram-4-IQ4_M \
    --precision mixed_3_8 --bulk-bits 4 \
    --te-imatrix imatrix.safetensors --dit-imatrix imatrix.safetensors

FP8 is the source every mirror is built from — the NF4 release can only be unpacked by bitsandbytes, which needs a CUDA host. The modulation and conditioning projections stay at 8-bit in every policy: adaln_modulation emits a per-layer scale and a tanh gate, so its error is multiplicative over the whole residual stream rather than averaged into one projection. The embedding, timestep and final-layer projections stay bf16 (~60 MB, and it removes them as suspects). Weights, tokenizer and chat framing are otherwise verbatim from the base repo.

License

Ideogram's non-commercial model agreement, inherited from the base weights — a quantized mirror is a Model Derivative under that agreement. LICENSE.md is included here verbatim; read it before using this.