MiniMax-H3-Single-Frame-VAE-500K

MiniMax H3 Single-Frame VAE 500K

This is an independent, experimental decoder-only checkpoint for decoding one image from a MiniMax H3 latent slice. It is not a full MiniMax H3 model, it does not include an encoder or transformer, and it is not used in production.

The decoder started from Mamad8/MiniMax-H3-Image-VAE and was further trained on 500,000 unique image-reconstruction examples using the frozen H3-compatible encoder. The full decoder and post_quant_conv were trainable. Text and captions were not used during decoder training.

Scope

Use this checkpoint when the intended output is one independently decoded image. It is most reliable on structured content such as product contours, line art, diagrams, documents, and UI-like layouts.

Do not treat it as a replacement decoder for MiniMax H3 video. Still-image training supervised one temporal boundary condition, while the full temporal decoder weights were updated. Later frames in a full-sequence decode can show grid or block artifacts, flicker, abrupt transitions, and texture drift.

Examples

All decoder comparisons use the same input image or the same frozen generated latent for every displayed decoder. The new T2I prompts were fixed before the outputs were reviewed.

Reconstruction

The fixed benchmark also includes natural photographs and scene text. Their aggregate metrics are reported below, but the source images are not redistributed here. A procedurally generated high-frequency stress case and a native H3 single-frame comparison are included.

H3 text-to-image through a single latent slice

The H3 t2va workflow generated a full normalized latent sequence. One fixed temporal slice was then independently decoded with the community decoder and this checkpoint.

Prompt: “Studio product photograph of a compact futuristic electric espresso machine, brushed aluminum and matte black, precise industrial design, centered three-quarter view, soft gray seamless background, crisp edges, realistic materials, no people, no text.”

Additional fixed examples:

• Industrial-design board
• Natural photograph

These examples test the decoder, not a native text-to-image pipeline. H3 still constructs and denoises its joint video/audio latent, so this is substantially more expensive than a purpose-built image generator.

First-frame-conditioned image editing

The following examples use H3's fl2va workflow. The contact sheets show the source followed by independently decoded temporal slices. They intentionally show the transition instead of selecting only the most flattering frame.

Instruction: “Change only the sofa upholstery from rust red to deep navy blue velvet. Preserve the room layout, window, lamp, rug, floor, lighting, and camera.”

Additional edits:

• Sketch-to-render transition
• Product-material transition

This is image-conditioned video used as a static-edit experiment. Early slices usually reconstruct the source, later slices apply more of the requested edit, and composition, lighting, material, or geometry can drift during the transition. A useful result may depend on searching multiple slices. That is a fundamental limitation compared with a native image-edit model.

Fixed hard suite: 8 T2I + 2 edits

This additional suite was preregistered before generation. It uses eight fixed T2I prompts and seeds plus two fixed source-conditioned edits. There was no seed search, prompt rewrite after review, or best-frame selection, and the failures are intentionally retained.

The eight outputs from our 500K decoder, without comparison framing:

For the controlled decoder comparison below, each pair decodes the exact same generated H3 latent:

The 500K decoder is consistently much sharper and more coherent than the community decoder across natural detail, technical diagrams, product detail, architecture, repeated texture, and dense UI. Exact text remains a clear failure mode: package copy, diagram labels, signs, and UI headings are often incorrect even when the surrounding structure is strong.

Both edits apply the requested color or material change. They also expose the main FL2VA limitation: stronger edit slices increasingly alter typography, lighting, controls, background objects, and other content that should have been preserved.

See the case-by-case visual review, the fixed prompt and seed registration, and the path-sanitized checksum manifest.

Fixed reconstruction benchmark

The benchmark contains 288 held-out cases spanning photographs, text, documents, diagrams, line art, product contours, UI-like content, and synthetic high-frequency stress patterns.

This checkpoint has the highest PSNR in this three-way benchmark, but the official H3 decoder remains better on SSIM, LPIPS, DISTS, and Edge F1. The result should not be read as universal superiority over the official decoder.

What the aggregate metrics mean

• PSNR emphasizes average pixel accuracy. This checkpoint is +0.7641 dB above official H3 and +8.4163 dB above the community initialization.
• SSIM measures structural similarity. Official H3 remains ahead by 0.0050.
• LPIPS and DISTS are perceptual-distance metrics. Official H3 remains clearly better, especially on photographs and small text.
• Edge F1 measures alignment of detected edges. This checkpoint is close to official in aggregate, but the result varies substantially by domain.

Compared with the community initialization, this checkpoint reduces LPIPS by about 72.8%, reduces DISTS by about 64.6%, and raises Edge F1 from 0.4647 to 0.7236. Compared with official H3, it trades slightly better average pixel accuracy for worse perceptual texture and slightly worse edge alignment.

Per-domain study

Each cell below is this checkpoint / official H3. Lower is better for LPIPS and DISTS; higher is better for the other metrics.

The strongest result is on diagrams, where this checkpoint beats official H3 on all five metrics. It also performs well on product contours, line art, documents, and UI. These domains reward clean geometry, flat regions, and structured edges, which match the decoder's training signal well.

The clearest weakness is natural photography. Pixel values remain close, but LPIPS and DISTS show that texture, tone, and fine local structure drift more than with official H3. Scene text is another weak area: small lettering and thin glyphs may soften, change shape, or become illegible. Dense grids, checkerboards, and repeated high-frequency patterns remain phase-sensitive and can show moiré or small line shifts.

Scaling from the earlier 75K decoder to 500K improved SSIM, LPIPS, DISTS, and Edge F1, but it did not close the photograph or scene-text gap. This suggests that additional generic reconstruction data alone is unlikely to solve those weaknesses. Targeted natural-texture, tiny-text, and phase-aware training would be more directly relevant.

An exact 74-frame native H3 latent panel also completed with no non-finite or geometry failures. Those generated latents have no ground-truth image, so decoder-to-decoder distances are disagreement measurements rather than quality scores. An AI visual review found the 500K outputs consistently sharper and more coherent than the community decoder, but this was not a human preference study and does not establish superiority over official H3 on generator latents.

Study boundaries

• Reconstruction metrics use 288 fixed held-out cases with known targets.
• T2I and editing examples are illustrative fixed-seed cases, not a benchmark.
• Generator-latent comparisons have no ground-truth image.
• The public study does not include human preference ratings or confidence intervals.
• The 1024 stage ran near the end of a cosine schedule with almost zero learning rate, so it does not establish mature 1024-resolution specialization.

Training

• Optimizer updates: 62,500
• Effective batch size: 8
• Learning rate: 2e-6
• Warmup: 625 updates
• Schedule: one continuous cosine decay
• Trainable modules: full decoder and post_quant_conv
• Frozen modules: encoder and quant_conv
• Distillation coefficient: 0.0

The reconstruction objective was:

1.0          * Charbonnier reconstruction
0.2          * first-order edge reconstruction
0.1          * (1 - SSIM)
0.05         * log-frequency-magnitude reconstruction
0.0113518046 * LPIPS-Alex

The final 5,000-image 1024 stage occurred near the end of the cosine schedule, when the learning rate was almost zero. It was neutral to slightly positive, but it does not establish meaningful 1024-resolution specialization.

Training data

Of the 500,000 training examples, 475,140 came from disjoint shards of Spawning/pd12m-full. The other 24,860 were native-resolution hard cases: 19,860 at 768 and 5,000 at 1024, balanced across WebSight, DocLayNet, Quick, Draw!, deterministic procedural diagrams, and deterministic procedural product contours. No training images are included in this repository. Users should review each source dataset's current terms and attribution requirements independently.

Loading

Clone the repository and install the pinned dependencies:

git clone https://huggingface.co/iamkaikai/MiniMax-H3-Single-Frame-VAE-500K
cd MiniMax-H3-Single-Frame-VAE-500K
pip install -r requirements.txt

The following CUDA example loads one normalized H3 generator-latent sequence, selects a temporal slice, decodes it independently, and writes a PNG. The input safetensors file must contain a tensor named latents with shape [batch, channels, time, height, width] in H3's normalized generator-latent space.

import torch
from diffusers import AutoencoderKLMiniMaxH3
from PIL import Image
from safetensors.torch import load_file

from load_decoder import load_single_frame_decoder

device = "cuda"
dtype = torch.float16

vae = AutoencoderKLMiniMaxH3.from_pretrained(
    "MiniMaxAI/MiniMax-H3",
    subfolder="vae",
    revision="42ed227ee7df40d41602854ae760620d6eb651fe",
    torch_dtype=dtype,
).to(device).eval()

load_single_frame_decoder(vae, "minimax_h3_single_frame_decoder_500k.safetensors")

# This is the normalized latent emitted by H3 generation, before applying
# vae.config.latents_std and vae.config.latents_mean.
normalized = load_file("normalized-latents.safetensors")["latents"]
latent_index = 0
z = normalized[:, :, latent_index : latent_index + 1].to(device, dtype=dtype)

latent_mean = torch.tensor(
    vae.config.latents_mean, device=device, dtype=dtype
).view(1, -1, 1, 1, 1)
latent_std = torch.tensor(
    vae.config.latents_std, device=device, dtype=dtype
).view(1, -1, 1, 1, 1)
z = z * latent_std + latent_mean

# Direct decoder use is intentional for a temporal extent of one.
with torch.inference_mode(), torch.autocast("cuda", dtype=dtype):
    decoded = vae.decoder(vae.post_quant_conv(z))

pixel_mean = torch.tensor(
    [0.485, 0.456, 0.406], device=device
).view(1, 3, 1, 1)
pixel_std = torch.tensor(
    [0.229, 0.224, 0.225], device=device
).view(1, 3, 1, 1)
frame = (decoded[:, :, -1].float() * pixel_std + pixel_mean).clamp(0, 1)

rgb = frame[0].mul(255).round().to(torch.uint8).permute(1, 2, 0).cpu().numpy()
Image.fromarray(rgb, mode="RGB").save("decoded-frame.png")

The file contains 585 tensors with decoder. and post_quant_conv. prefixes. It intentionally contains no encoder weights. load_single_frame_decoder strictly verifies the 9 GB file's SHA-256 by default, so the first load can take several seconds before GPU inference begins.

Known limitations

• Single-frame use only. Full video decoding is unsupported.
• Natural textures can be smoothed or shifted.
• Small lettering and thin glyphs can change.
• Dense lines, checkerboards, and fine grids can show moiré or phase errors.
• H3 T2I and FL2VA examples still pay the cost of full video/audio latent generation.
• FL2VA edits can require searching temporal slices and can drift in crop, lighting, material, geometry, or unrelated content.
• The public comparisons are a bounded benchmark, not a human preference study and not evidence of universal superiority.

Reproducibility

• Training exposures: 500000
• Optimizer step: 62500
• Training config fingerprint: 225279cbb4cd1e2db94b5b2ae15e123a921fefada0782e5c68d35302e4b4d0a9
• Original final decoder SHA-256: 4a5e4d0041c6fdd15af5292db3e10574e28500b216ad2d1f78b5951f2b8abfe2
• Audited release export SHA-256: 6c5ff2caa8fade6769f4dd53ee244f77a06652c8cb66b2fedc93f75046d9f001
• Community initialization revision: c7b9252c73707dba494cf4d99ca45d3f33f561b3
• Official MiniMax H3 revision: 42ed227ee7df40d41602854ae760620d6eb651fe

The release export contains the same 585 decoder tensors as the exact final resume state. Its file hash differs from the training export because the release file includes provenance metadata.

License

This checkpoint is a modified derivative of MiniMax H3 and remains subject to the MiniMax H3 Community License Agreement. The upstream license contains territorial and use restrictions. Review it before downloading, using, or redistributing this checkpoint. See NOTICE and MODIFICATIONS.md.