Downloads · 30 days
20
23% of all-time downloads
moonzerokevin/qwen3vl-32b-minimax-h3-nf4
qwen3vl-32b-minimax-h3-nf4 is a image-text-to-text model from moonzerokevin. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
The text encoder used by MiniMaxAI/MiniMax-H3, quantised to 4-bit NF4 with bitsandbytes so it fits on a single 32 GB card.
Downloads · 30 days
20
23% of all-time downloads
All-time downloads
86
Public
Parameters
33.4B
19.5 GB on disk
Likes
1
Public
Click a slice to open those files.
.safetensors19.5 GB · 100%
How the weights are stored.
U831.8B · 95%
From the Hugging Face model README
The text encoder used by MiniMaxAI/MiniMax-H3, quantised to 4-bit NF4 with bitsandbytes so it fits on a single 32 GB card.
The H3 repo ships the stock Qwen/Qwen3-VL-32B-Instruct as its text encoder — we
verified this (1058 tensors, 66.7 GB, identical key structure), so this artifact is just a
quantisation of the official Qwen weights, nothing H3-specific.
In bf16 the encoder is 66.7 GB, which does not fit on a 5090/4090-class card. The usual
workaround is accelerate.cpu_offload, which streams the whole model across PCIe on every
request. In a three-stage H3 deployment that made text encoding the pipeline bottleneck.
| bf16 + cpu_offload | this artifact (NF4 resident) | |
|---|---|---|
| text encode per request | 7.4 – 8.5 s | 0.09 s (text-only) / 0.5 – 1.5 s (with a keyframe) |
| weights on disk | 66.7 GB | 19 GB |
| VRAM while resident | 1.5 GB (weights keep crossing PCIe) | 18.2 GB |
| load time | ~16 s (read 66.7 GB + quantise) | ~6 s |
Measured on an RTX 5090 (31.36 GiB), transformers 5.15.0, bitsandbytes 0.50.1.
prompt_embeds versus the bf16 reference: cosine 1.00003, relative L2 1.6 %.
End-to-end, generating video with the same prompt, spec and seed — only the encoder differs — frame PSNR against the bf16 run:
| comparison | PSNR |
|---|---|
| same file against itself | ∞ |
| bf16 ↔ NF4, first+last-frame conditioning | 24 – 29 dB |
| bf16 ↔ NF4, first-frame conditioning | 18 – 28 dB |
| bf16 ↔ NF4, text-to-video | 15 – 17 dB |
| same config, different seed | 8.6 – 10.2 dB |
The encoder swap sits far from the "different seed" floor, i.e. it perturbs the same clip rather than resampling a new one — and image-conditioned modes are affected least, because the reference frame anchors the composition.
from transformers import Qwen3VLForConditionalGeneration
text_encoder = Qwen3VLForConditionalGeneration.from_pretrained(
"moonzerokevin/qwen3vl-32b-minimax-h3-nf4", device_map="cuda")
Then hand it to the H3 modular pipeline in place of the bf16 encoder:
pipe = ModularPipeline.from_pretrained(h3_path, workflow="fl2va")
pipe.load_components(names=["tokenizer", "processor", "vae", ...]) # skip text_encoder
pipe.text_encoder = text_encoder
Requires bitsandbytes (tested with 0.50.1 on sm120).
RedHatAI/Qwen3-VL-32B-Instruct-NVFP4 (compressed-tensors, ~16 GB) loads under
transformers but cannot run inference there — the NVFP4 kernels live in vLLM; the
forward pass fails with 'Linear' object has no attribute 'weight'. That is why this
artifact uses bitsandbytes NF4 instead.bnb_4bit_use_double_quant=True), compute dtype bfloat16.