MiniMax-H3-NVFP4

MiniMax H3 — NVFP4

NVFP4 quantizations of the MiniMax-H3 diffusion transformer for ComfyUI — both task variants, ref2va (reference images → video) and fl2va (first/last frame → video).

• Original model: https://huggingface.co/MiniMaxAI/MiniMax-H3
• ComfyUI repackage these derive from: https://huggingface.co/Comfy-Org/MiniMax-H3

NVFP4 requires an NVIDIA Blackwell GPU (RTX 50-series, RTX PRO 6000, B200). On Ada, Hopper or older the NVFP4 path is emulated — use Comfy-Org's int8_convrot files instead.

Diffusers / Chutes torchao DiT (this fork)

This fork of lilcheaty/MiniMax-H3-NVFP4 adds:

minimax_h3_fl2va_pruned_nvfp4_torchao.safetensors (~12 GB)

• Same pruned fl2va NVFP4 weights as lilcheaty’s Comfy file, repacked for torchao TensorCoreNVFP4StorageLayout so Diffusers MiniMaxH3Transformer3DModel can keep the DiT GPU-resident on Blackwell.
• Not loadable in ComfyUI — use lilcheaty’s minimax_h3_fl2va_pruned_nvfp4.safetensors for Comfy.
• Source / reference quantizations: https://huggingface.co/lilcheaty/MiniMax-H3-NVFP4
• Upstream MiniMax: https://huggingface.co/MiniMaxAI/MiniMax-H3

Runtime notes for the Chutes/Diffusers path: AdaLN must match Comfy’s pruned layout (apply_silu=False, table lerp, fp32 AdaLN island) and SwiGLU must use Comfy gate|up packing — loading this file into stock Diffusers without those patches produces tiled noise.

Which file do I want?

First pick the task, then the file:

• ref2va — up to 9 reference images (+ videos/audio) → video. Identity-driven.
• fl2va — first and/or last frame → video. Interpolates between endpoints; also how you chain clips past one generation by feeding the previous clip's last frame in.

Take a pruned_nvfp4 file when footprint and speed matter: half the size of the alternatives at equal-or-better speed, with the modulation path left at full precision. If you have VRAM to spare and want maximum fidelity, read "Honest limitations" first — 4-bit weights appear to cost some motion quality relative to Comfy-Org's 8-bit int8_convrot.

The two _convrot_int8 files are not ours — they are rockerBOO's work, mirrored here for convenience under the same MiniMax community license. A mixed-precision profile quantized in a single pass from bf16: NVFP4 for blocks.2-46 mlp, FP8 for attn.qkv_proj/adaln_proj, BF16 left on the layers stored F32 upstream, and attn.qkv_proj swapped to INT8 ConvRot quantized fresh rather than restacked. Bigger than ours, better motivated on fidelity. Original repo: https://huggingface.co/rockerBOO/minimax-h3-nvfp4

The two pruned files are the same size and were produced by the same script over the same 200 quantized layers — they differ only in task head. fl2va s/it is unmeasured but should match ref2va; the architectures are identical.

Why the pruned base is the right one to quantize

Comfy-Org's pruned checkpoint is not lossily pruned — it is a structural refactor of AdaLN, and understanding it explains the whole table above.

In the bf16 model, AdaLN modulation dominates the parameter count:

The bf16 model projects a 5376-dim conditioning vector into modulation parameters per block. The pruned model replaces this with an 8-dim timestep table (adaln_t_table, shape [1025, 8]) feeding adaln_proj.linear of shape [96768, 8]. Because modulation depends only on the timestep, that 5376-wide projection was almost entirely redundant — 13.04B parameters collapse to 0.04B, a ~326x reduction.

This matters for quantization because AdaLN is the part you least want to quantize: it emits the scale and shift applied to every residual stream, so error there is multiplicative and compounds across all 50 blocks and every sampling step. In the bf16 model you face a bad choice — protect AdaLN and produce a ~36 GB file (larger than the 34 GB int8 it should beat), or quantize 39% of the model and hope. In the pruned model the problem disappears: AdaLN is already tiny, so you keep it at full precision for free and quantize only attn+mlp, which are error-tolerant.

Comfy-Org's pruned_int8_convrot quantizes exactly those 200 attn/mlp layers to int8_convrot and leaves everything else alone. pruned_nvfp4 takes that same set to NVFP4.

Measured

RTX PRO 6000 Blackwell (96 GB), ComfyUI 0.30.0, ref2va, 864x480, 39 frames, 20 steps, res_multistep / beta, three matched seeds. These are speed/size numbers only — the quality comparison from the same runs is retracted (see "Honest limitations"). Note also that beta was the wrong scheduler; ComfyUI's official H3 templates all use simple.

-40% file size, -8.0 GB VRAM, -12.4% sampling time.

At ~12 GB for the DiT, a 32 GB card (RTX 5090) becomes viable if the text encoder is offloaded to CPU after encoding — it runs once per prompt, not once per sampling step.

How pruned_nvfp4 was built

Single pass from pruned_bf16. Comfy-Org's minimax_h3_{ref2va,fl2va}_pruned_bf16 (40.2 GB) is full-precision weights with AdaLN already refactored into the timestep table, so the 200 attn/mlp layers are quantized once, straight from bf16:

nq, nparams = ckt.TensorCoreNVFP4Layout.quantize(w.contiguous())      # bf16 -> NVFP4
tensors = ckt.TensorCoreNVFP4Layout.state_dict_tensors(nq, nparams)

The layer set is blocks.<N>.{attn.qkv_proj, attn.out_proj, mlp.fc1, mlp.fc2}.weight — 50 blocks x 4 = 200, verified set-identical to the layers Comfy-Org quantize in their pruned_int8_convrot release. Everything else (AdaLN, norms, embedders, token_refiner, all biases) is copied verbatim at source precision: 332 tensors, 1132 in the output.

Per-layer config lives in a comfy_quant uint8 tensor holding JSON, written as {"format": "nvfp4"}; there is no file-level __metadata__.

Full script: bf16_to_nvfp4.py in this repo. 200 layers in ~5 seconds on one GPU.

<details> <summary>Earlier builds requantized from int8 (superseded)</summary>

Before Comfy-Org published a pruned bf16 checkpoint, the only pruned source was pruned_int8_convrot, so these files were built bf16 -> int8_convrot (Comfy-Org) -> NVFP4 — two lossy passes whose error compounds. That path needed a real dequantize -> requantize, because the StarNodes model converter passes non-floating-point tensors through untouched:

if not tensor.dtype.is_floating_point:
    return tensor          # already-int8 weights are copied verbatim

so running it on an int8 checkpoint silently produced a byte-identical file. The old script (pruned_to_nvfp4.py) is kept for reference. Both files were rebuilt single-pass on 2026-08-05; the output key set and dtype census are unchanged, only the error path is.

</details>

Prompting: H3 wants a structured IR, not prose

Read this before blaming the weights for bad output. H3 was trained on the structured output of H3-Context-IR, a preprocessing model that rewrites a plain request into labelled sections; MiniMax's model card calls it "critical to the quality of the final output". ComfyUI passes your raw string straight to the DiT, so you must write that structure yourself.

Official guides: base · ref

For the target video, at 0.00 seconds into the target video, <Picture 1> (from [Shot 1]) is fully referenced.

integrated_multimodal_description: [Shot 1] Live-action, cinematic, the young woman shown in
<Picture 1> remains beside the rain-covered train window, preserving her appearance and the
carriage layout. The camera trucks right with small amplitude at slow speed as she lifts her
gaze toward the passing city lights. The quiet, breathy young woman (S1) says:
<d>[English] I get off at the next station.</d> She folds the letter along its existing crease.

overall_soundscape: The train wheels produce a steady metallic rhythm beneath a low
ventilation hum. Rain ticks against the window while paper rustles softly in her hands.

non_diegetic_music: Sustained cello notes at a slow tempo with widely spaced piano tones.

Dialogue must be explicit or you get gibberish. Speech is generated jointly with video, so saying that someone speaks without giving the words yields correct prosody and mouth shapes with no lexical content. Speaker identity, action and delivery go outside <d>; only the language tag and verbatim words go inside. Use stable IDs (S1), (S2), and (S1,S2) for simultaneous speech.

Other essentials: [Shot 1] carries no timestamp, later shots use [Shot N] At MM:SS.mmm; aim for 350-500 words of description; write camera motion as type + amplitude + speed; reference tags must appear in the order the inputs were connected. ref2va accepts up to 9 reference images, and 3-4 varied shots hold identity far better than one.

Companion files (mirrored, not ours)

The NVFP4 text encoder is Comfy-Org's work, not ours. Only the minimax_h3_*_nvfp4*.safetensors diffusion models here are new.

VAEs are deliberately not quantized: they are small, run once per generation rather than per step, and decode straight to pixels and audio samples where error is immediately visible. The text encoder being NVFP4 buys VRAM, not speed — it also runs once per prompt.

Usage

📂 ComfyUI/models/
├── 📂 diffusion_models/
│   └── minimax_h3_ref2va_pruned_nvfp4.safetensors   (or the fl2va file)
├── 📂 text_encoders/
│   └── qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors
└── 📂 vae/
    ├── minimax_h3_video_vae_fp16.safetensors
    └── minimax_h3_audio_vae_fp32.safetensors

Recommended stack total: 33.7 GB. Use the official R2V template, swapping the diffusion model — or the FL2V template for the fl2va file. Requires ComfyUI >= 0.30.0 (native H3 support in comfy/ldm/minimax/). CLIPLoader type must be minimax; sampler res_multistep; frame length must satisfy 17n+5.

Honest limitations

• NVFP4 is 4 bits/weight and that appears to cost visible quality vs int8_convrot. In side-by-side playback of 15s clips at 1152x640, the int8 build showed noticeably less mid-motion artifacting and held object shape better through fast pans. If you have the VRAM (~20 GB vs ~12 GB) and can spend ~14% more time per step, Comfy-Org's pruned_int8_convrot may be the better choice. This repo's value is smallest-footprint and fastest, not highest-fidelity.
• That observation predates the single-pass rebuild and has not been repeated since. It was made against the doubly-quantized files, so some of the smearing may have been compounding error rather than 4-bit weights. Treat the int8-is-better lead as untested against the current files.
• That comparison is not controlled: the two runs also differed in text encoder (nvfp4_awq vs int8_convrot), it was n=1, and matched seeds do not produce matched trajectories across different quantization. Treat it as a lead, not a result.
• The earlier claim of "no visible degradation across three matched seeds" is retracted — it was measured at 480x864 / 39 frames, too small and too short to show what appears at 768p over 15s. Contact sheets in particular hide motion artifacts; judge on playback.
• 15s (362 frames) generates fine; see the VRAM note below.
• The fl2va file is unbenchmarked — it was produced by the same script over the same layer set as ref2va, but has not been run end-to-end.
• Benchmarks are single-GPU, one card, one resolution, and were measured on the older doubly-quantized files. Size and layout are unchanged by the rebuild, so s/it and VRAM should carry over, but they have not been re-measured.

Failure cases are welcome in the discussions tab — concrete artifacts beat aggregate scores.

VRAM / length ceiling

Measured on RTX PRO 6000 (96 GB), NVFP4 pruned DiT, 20 steps. The budget that matters is pixels x frames, not resolution alone:

The ComfyUI template default (1344x768) is safe at its default 124 frames but not at 362. OOM here is a SIGKILL: no traceback, no /history entry, and ComfyUI restarts with its queue wiped. A job that silently vanishes from /queue is this.

License

Inherits the MiniMax-H3 Community License Agreement from the original model.