Downloads · 30 days
131
2% of all-time downloads
abenliao/minimax-h3-transformer-mxfp8
minimax-h3-transformer-mxfp8 is a machine learning model from abenliao. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for diffusers.
e4m3 with one UE8M0 exponent per 32 elements, in cuBLASLt's swizzled 128x4 scale tiling. Unlike an e4m3-plus-float32-scale checkpoint, this one is not expanded before the GEMM — the Blackwell tensor cores multiply it…
Downloads · 30 days
131
2% of all-time downloads
All-time downloads
5.7K
Public
Parameters
33.9B
34.3 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors34.3 GB · 100%
How the weights are stored.
F8_E4M333B · 98%
From the Hugging Face model README
e4m3 with one UE8M0 exponent per 32 elements, in cuBLASLt's swizzled 128x4 scale tiling. Unlike an e4m3-plus-float32-scale checkpoint, this one is not expanded before the GEMM — the Blackwell tensor cores multiply it directly.
Exported from the stock MiniMaxAI/MiniMax-H3 transformer/. 66.3 GB → 34.3 GB.
adaln_proj.linear.weight is deliberately left in the block-scaled float32
format: its GEMM folds the bias in with beta = 1, which rounds once where a
separate add rounds twice, and its m is one or three rows, so FP8 tensor cores
would buy nothing. The loader picks the format per tensor from the scale suffix
beside it (.mx_scale vs .fp8_scale), so the mix needs no flag.
| e4m3 storage, bf16 GEMM | this | |
|---|---|---|
| per step, 5 s / 37,738 rows | 43.0 s | 35.0 s |
| 16 steps, end to end | 822 s | 661 s |
| peak VRAM | 7.35 GB | 9.50 GB |
1.23x, not the 3.3x the GEMM alone measures. Attention is more than half the floating point at this length and is a separate bf16 kernel that this does not touch. The extra 2.15 GB is the quantised activation and its scales.
Against the same seed through the e4m3 path: 4.01% of pixels identical, mean 19.22 levels apart. The yardstick for "two correct implementations" on this model is 1.64 levels, so this is a real cost and not rounding.
Do not use this format for the conditioner. Measured there at 13.5 s → 33.9 s:
28 rows makes every GEMM memory-bound, so the tensor cores idle while the
per-call activation quantisation is pure added work. Use
abenliao/minimax-h3-conditioner-fp8 for that half of the pipeline.
cublasLtMatmul returns NOT_SUPPORTED on anything without FP8 tensor cores,
which is the correct answer rather than a bug. On an A6000 or an H100 this
checkpoint will not run; re-export from the bf16 source with H3_MX=0.