3d-mini-wham-long-horizon

3D Mini-WHAM long-horizon checkpoints

Full-resume checkpoints from the 23-second 3D dungeon-crawler architecture search in world_model_experiments PR 25.

Each resume.pt includes model weights, AdamW optimizer state, random-number-generator state, sampler/window state, exact run specification, tokenizer/data provenance, cumulative FLOPs, and spend accounting. The frozen tokenizer and pre-encoded dataset are published separately at osazuwa/3d-mini-wham-tokenizers and osazuwa/3d-mini-wham-encodings.

Runs

• lh-XL-causal-12000: 170,981,672 trainable world-model parameters.
• lh-XXL-causal-6000: 363,604,008 trainable world-model parameters.

Both retain the trace-prediction, DSEP, and event-timing objectives. Milestone and final checkpoints are stored under runs/<run>/checkpoints/step-<step>/.

Final architecture-search result

Training architecture winner: XL (170,981,672 parameters, step 12,000). On the original four held-out 23-second episodes, its discrete-token rollout achieved 19.035 PSNR / 0.545 SSIM over all 226 generated frames, versus 17.121 / 0.462 for freezing the last observed frame. XL reduced mean squared pixel error by 35.6% and reached the final green exit qualitatively.

XXL (363,604,008 parameters, step 6,000) was stronger for the first few seconds but became stuck around the chest. At the full horizon it achieved 16.735 / 0.508, 9.3% more pixel error than the frozen-frame baseline.

Watch the 32 held-out rollouts

The follow-up evaluation generated 32 deterministic held-out episodes. They are presented as two 4x4 video grids rather than summarized by an analysis figure:

• Episodes 1-16
• Episodes 17-32

Each tile is labeled with its episode ID and shows ground truth | XL. Both grids play the complete 23 seconds at 10 fps. The first four frames (0.4 seconds) are observed context; after that, the right side is generated autoregressively while ground-truth actions are replayed. These grids show the eight-step discrete MaskGIT token output from the uncoupled Mini-WHAM, not a coupled q + residual render.

How to read them: follow scene progress, not just pixel sharpness. A healthy rollout should move from the opening corridor through the door/chest sequence and later traversal to the green exit. The characteristic failure is an attractor: the prediction remains around the early door or chest while ground truth continues. Also watch for repeated geometry, motion without stage progress, and an exit appearing at the wrong time. Because every episode follows the same scripted room sequence, variation across the grid measures reliability within this task rather than broad gameplay generalization.

Supporting aggregate measurements

With eight MaskGIT refinement steps, XL averaged 19.470 PSNR / 0.569 SSIM and beat frozen-frame persistence on 30/32 episodes. The automated progress estimate placed 10/32 endpoints in the exit sequence and 16/32 in the early door/chest sequence; treat those counts as supporting diagnostics for what is visible in the grids.

Selected inference setting: greedy six-step MaskGIT. Against the same episodes, six steps:

• averaged 19.905 PSNR / 0.582 SSIM;
• improved paired full-horizon PSNR by 0.436 dB (95% bootstrap CI 0.074 to 0.828) and SSIM by 0.0138 (0.0014 to 0.0273);
• improved PSNR on 23/32 episodes and beat persistence on 31/32;
• placed 14/32 endpoints in the exit sequence, while the green-exit detector was present at the endpoint in 12/32 versus 10/32 with eight steps;
• retained seven detected exits, gained five, and lost three.

This is not universal monotonic progress: six episodes ended in an earlier estimated stage and three eight-step exits were lost. The paired image metrics and net exit result nevertheless favor six steps at this sample size.

Play the 23-second exit-transition grid (truth | 8-step | 6-step in each tile).

Inference throughput

Clean single-stream mixed-precision measurements on one NVIDIA A100-SXM4-80GB were:

Six steps therefore crosses the 10 fps target on average while preserving quality in the completed 32-episode study. This does not yet establish production real-time interaction: p95 action-to-photon latency, input and display overhead, and sustained serving behavior were not measured.

The current rollout recomputes the full ten-frame temporal window for every generated frame and does not support KV caching. The next structural speed target is an in-distribution rolling-window KV cache with ten-frame trimming and temporal-position rebasing, followed by quality-parity and p95 latency validation. Compiler and CUDA-graph work can follow; the unused DiT path is not on the selected MaskGIT deployment path.

Artifacts

• XL full checkpoint: runs/lh-XL-causal-12000/checkpoints/step-012000/resume.pt
• XXL full checkpoint: runs/lh-XXL-causal-6000/checkpoints/step-006000/resume.pt
• 32-episode eight-step analysis
• Six-step paired comparison
• Single-stream latency benchmark
• All 32 six-step rollout videos
• XL report and 24 original videos: osazuwa/3d-mini-wham-evaluations/evaluations/61380741c3b652828e3aa8d3243af69d11170e5b937052d6cb4b56f0ed64d799
• XXL report and 24 original videos: osazuwa/3d-mini-wham-evaluations/evaluations/86028b412d0ec8bdee1411f5b5831f20c2eb55ea703eb357fa44d4f3d4c92cfc