Mira-Scene

Mira-Scene

Mira-Scene reconstructs an editable 3D scene from a single image. The full pipeline combines instance segmentation, monocular depth estimation, canonical coordinate map (CCM) and occupancy prediction, per-object mesh generation, support-aware scene assembly, and environment-map generation.

The project code and complete inference instructions are available in the Mira-Scene GitHub repository.

Available pipelines

This repository contains two CCM checkpoints:

• pipeline/: the released checkpoint intended for Mira-Scene inference;
• pretrain_pipeline/: the stage-1 pretraining checkpoint intended to initialize the second-stage training configuration.

Both are Diffusers-compatible CCMVoxelPipeline directories. They predict a pixel-aligned canonical coordinate map and a sparse voxel representation for each segmented object. Neither checkpoint is a standalone image-to-GLB model: a scene image and per-instance masks are prepared by the segmentation stage, and supported mesh and scene-construction stages consume the CCM outputs.

The pipelines contain the image encoder, flow-matching scheduler, CCM/voxel diffusion transformer, sparse-structure VAE, and image preprocessor configuration.

Download the released inference pipeline

Install and authenticate the Hugging Face client if necessary:

python -m pip install -U huggingface_hub
hf auth login

Download the released pipeline:

hf download Yang-Tian/Mira-Scene \
  --include "pipeline/*" \
  --local-dir checkpoints/mira-scene

Set checkpoints.ccm in infer_scripts/config/local.yaml to:

checkpoints:
  ccm: /absolute/path/to/checkpoints/mira-scene/pipeline

Download the pretrain checkpoint

The pretrain checkpoint can be used as the initialization for the second-stage finetuning described in example_train/README.md:

hf download Yang-Tian/Mira-Scene \
  --include "pretrain_pipeline/*" \
  --local-dir checkpoints/mira-scene

Point system.params.pretrained_model_name_or_path in example_train/configs/finetune.yaml to:

system:
  params:
    pretrained_model_name_or_path: /absolute/path/to/checkpoints/mira-scene/pretrain_pipeline

The full inference pipeline also uses third-party segmentation, depth, and mesh checkpoints. See the environment and checkpoint guide for model IDs, download commands, and configuration keys.

License

Original Mira-Scene code, configurations, and model artifacts are released under the MIT License. Third-party components, base models, pretrained encoders, and backend checkpoints retain their respective licenses and terms of use.

Inference

After installing the stage-specific environments and configuring the selected backends, run:

python infer_scripts/pipeline.py \
  --input /path/to/image_or_directory \
  --output /path/to/results \
  --config infer_scripts/config/local.yaml

To run only the CCM stage on prepared cases:

python infer_scripts/2_inference_CCM.py \
  --demo_dir /path/to/prepared_cases \
  --output_dir /path/to/results \
  --ckpt_dir /absolute/path/to/checkpoints/mira-scene/pipeline \
  --use_cropped_condition

Refer to the project inference guide for the required input layout and the complete stage-by-stage workflow.