minimax-h3-deltaquant-x0-trajectory

Where quantization error enters the denoising trajectory

MiniMax-H3, NVFP4 DeltaQuant against a BF16 reference, compared along the 50-step trajectory rather than only at the end. Each sample point is the model's own clean-latent estimate (x0) — what it currently believes the finished video looks like — captured at that step and decoded.

480x864, 124 frames, seed 1101, 50 steps, no step cache on either arm (caching perturbs the trajectory too, and this experiment isolates the quantization effect). Three scenes: weld (robotic arm welding a car chassis), reentry (astronaut in a cockpit), cats.

Metrics are computed on latents, not decoded frames — at these fidelities H.264 noise would dominate. x0 = x_t + (1−t)·v; the + matters, H3's transformer predicts x0 − eps, the opposite of the usual convention, and a minus sign yields something plausible-looking and wrong. Verified bit-exact (rel_l2 = 0.0, max_abs = 0.0) against the pipeline's own final latent in every one of 10 runs.

The shape of the divergence: a ramp over the first ~8 evaluations, then flat

Dense sampling on weld (evaluations 0,1,2,3,4,6,8,10 plus the last):

Half the divergence exists by evaluation 3, 91% by evaluation 6, and it saturates by evaluation 8. For the remaining 41 evaluations it does not move. A five-point sample makes this look like a step change at 10; the dense sample shows it is a ramp that finishes early.

rel-L2 above 1.0 means the difference exceeds the signal — the two arms are no longer near neighbours, they are different samples. So the flat tail is not "degradation that stopped accumulating", it is two coherent trajectories that separated and then each refined its own scene.

Replicated on all three scenes (rel-L2 at first evaluation → last): cats 0.145 → 0.442, weld 0.311 → 1.086, reentry 0.187 → 0.716. The first evaluation is always closest by 2.5–4.6x, the jump is always inside the first ten evaluations, then flat. Magnitude tracks how much high-frequency content the scene has; the timing does not vary.

A useful control: the self rel-L2 to final columns — how far each arm's own x0 is from its own final latent — stay within 1–3% of each other from evaluation 25 onward. The quantizer changes where the trajectory lands, not how fast it converges.

Sharpness is scene-dependent, not systematic

Two of three sharper, one softer — this does not generalise, and it reproduces the same sign flip seen at 768p in the companion repo. What is consistent is that the error is high-frequency-weighted: measured at evaluation 0, where both arms receive identical inputs so the difference is pure quantization error, the error's high-to-low frequency power ratio is 6.6x that of the image itself, and the error is 2.1x stronger at latent-cell boundaries (32 px, the VAE's spatial pitch) than in cell interiors. The cube pitch was tested separately and is not the periodicity.

That follows from the method's structure — the core is a cube mean, hence a low-pass, kept at FP8, while the delta (the high-pass residual) is what goes to 4 bits. The 4-bit error lands in the high-frequency band by construction, and the piecewise-constant core leaves steps at token boundaries where the delta cannot correct it exactly. Whether that reads as sharper or softer in pixels depends on what the scene's own high-frequency content was doing.

On weld the signature is directly visible at evaluations 0–1: DeltaQuant's x0 carries a woven high-frequency texture the reference does not, at 2.8–2.9x the Laplacian variance, on an image that contains almost nothing yet. That is the 4-bit error before the model has decided anything. This tell is weld-specific — on cats and reentry the first x0 is empty in both arms (Laplacian 1.5/0.6 and 2.3/2.2), so there is nothing to compare.

Files

Prefixed by scene; weld-dense__* is the densely-sampled run.

Raw x0 latents (rows plus a decodable latent per stage, written before any decode) are kept alongside the runs rather than uploaded — re-decoding is cheap, re-generating is not.

Correction to previously published speed numbers

Earlier artifacts in this series reported DeltaQuant at 2.42x over BF16. Withdrawn. The figure was inflated by a defect in the reference: our BF16 path used diffusers' enable_group_offload(use_stream=False), which never pins host memory and performs a synchronous device→host copy into freshly allocated pageable memory for every block on every step — 38.5 GB per step at a measured 2.27 GB/s, against 24.2 GB/s for a straight copy on the same card.

Pinning the host copies once and prefetching one block ahead takes the same BF16 run from 1901.9 s to 888.6 s — with bit-identical latents (torch.equal True, max_abs = 0.0). Scheduling changed; arithmetic did not.

Re-measured solo, zero neighbours, 1344x768 / 50 steps:

The quantized arms reproduced their older numbers to 1.5–2.0%, so contention was never the problem — the reference was. An independent SGLang run of the same model, prompt, seed and resolution on the same machine takes 1045.4 s, which the fixed path now beats.

Companion: yitongl/minimax-h3-deltaquant-nvfp4.