MiniMax-H3-W4A4

MiniMax H3 W4A4 (SVDQuant, nunchaku)

The 50 transformer blocks of MiniMax H3 quantized to int4, weights and activations, in the SVDQuant lineage. It is one of the first weights-and-activations int4 builds of H3 on pre-Blackwell GPUs, and the one running real int4 GEMM through nunchaku. It runs on one RTX 4090, and the venue target is an RTX 3090.

H3 is a 33B omni transformer that generates video and stereo audio in the same denoising pass. Its published paths want a 128GB unified-memory box. The other 4-bit ports on the Hub require a Blackwell GPU and quantize weights only, or mark their W4A4 variants experimental. This one quantizes weights and activations with real int4 GEMM kernels through nunchaku, calibrated on activations collected during real generation.

Numbers (RTX 4090, 124-frame 832x480 clips with stereo audio, same seed as the bf16-path reference)

Calibration of all 50 blocks took 24 minutes on the same card. Quality holds by eye across an 8-scene bench, trajectories diverge from the reference as any 49-step rollout does under perturbation, coherence and detail do not drop.

Files

transformer_blocks.safetensors (10.58 GB) holds the 300 quantized linears in the standard nunchaku two-file block format, qweight int4 group 64, wscales, rank-32 low-rank branch, smooth factors. Rows follow the diffusers layout of the root transformer/ of the H3 repo, to_q, to_k, to_v, to_out.0, ff.net.0.proj, ff.net.2 per block. Everything the checkpoint does not carry, AdaLN, norms, embedders, token refiner, heads and VAEs, is read from the original H3 repo at load time.

How to run

Loader, runtime, calibration pipeline and all the documented traps live in the companion repo, sztlink/h3-consumer-bench. The short version. Build the diffusers modular pipeline of H3 (PR 14355 branch), load the transformer int8, swap the 300 block linears for SVDQW4A4Linear from this checkpoint behind a 2D wrapper, precompute the AdaLN table from the deterministic sigma schedule, leaf-offload the 9.8GB video VAE. nunchaku 1.3.0.dev wheels cover torch 2.11.

One trap deserves the front page. The H3 repo ships two transformers. FL2VA/transformer is the reference sibling that never executes in the t2va and i2va paths. The runtime uses the root transformer/, different weights, same shapes. Calibrate against the wrong twin and every diagnostic stays green while the output is pure noise.

Lineage

Krea 2 Turbo W4A4 port, then krea-realtime-bench, then h3-consumer-bench, then this. Quantized weights are a derivative of MiniMax H3 and carry the MiniMax H3 Community License.

The work it came from

Quantized by Felipe Sztutman, as part of the research behind 2047: Arquivos do depois de antes, by Giselle Beiguelman and Felipe Sztutman, on view at Casa Firjan, Rio de Janeiro, from October 15 to November 16, 2026. A machine reads the real news on rare earth mining, writes a chapter dated 2047, prints it line by line on a dot matrix printer and projects scenes of it on the wall, generated on site, no cloud. The energy of every scene is measured on the card and published with each chapter at heretica.xyz (English).

This W4A4 build is not the one on the wall. Judged by eye on the installation footage it lost too much, so the show runs H3 with the 300 block linears in int8 weight-only, streamed block by block on an RTX 3090 (and an RTX 4090 for the backlog), the AdaLN collapsed into a 97 MB table with the turbo LoRA by larryvrh (Apache 2.0) baked in, and Sol-Attn sparse attention. The site tells how the scenes are made, in English.

More about the author at felipesztutman.com.

License and territory

The quantized weights are a derivative of MiniMax H3 and are distributed under the MiniMax H3 Community License Agreement, see NOTICE. The license grants rights only within its Applicable Territory, which today excludes the European Union, the United Kingdom, the Republic of Korea and the United States. By downloading or using these files you accept that agreement, including its territorial limits and its Acceptable Use Policy (Exhibit A). Outputs posted publicly must be disclosed as machine generated.