InSpace

InSpace: Structure-Aware 3D Indoor Scene Generation from a Single 360° Image

<p align="center"> <a href="https://arxiv.org/abs/2607.03990"><img src="https://img.shields.io/badge/Paper-Arxiv-b31b1b.svg" alt="Paper"></a> <a href="https://kookie12.github.io/InSpace-Project-Page/"><img src="https://img.shields.io/badge/Project-Website-blue" alt="Project Page"></a> <a href="https://github.com/kookie12/InSpace"><img src="https://img.shields.io/badge/Code-GitHub-black" alt="Code"></a> <a href="https://huggingface.co/datasets/GwanHyeong/ERP-FRONT-30K"><img src="https://img.shields.io/badge/Hugging%20Face-Dataset-orange" alt="Dataset"></a> </p>

Venue: ECCV 2026 · Malmö, Sweden

Introduction

InSpace generates a complete, asset-aware 3D indoor scene from a single 360° (equirectangular) panorama, producing a full-room mesh along with individual, separable, textured furniture meshes. It is built on the TRELLIS.2 O-Voxel representation and extends it with a panorama-native, structure-aware generation pipeline: view-selective cross-attention driven by the camera center, layout-guided structure inversion from monocular depth, a 3D bounding-box estimator, and asset-aware shape and texture generation with global-local hybrid attention.

Concretely, the scene is produced in three cascaded flow-matching stages: (1) estimating Partial Scene Geometry (PSG) as a spatial prior, (2) generating coarse scene structure (CSG) with view-selective cross-attention, and (3) producing detailed layout and asset geometry with textures through a global-local hybrid attention.

Model Details

• Model Type: Three-stage flow-matching framework on O-Voxel structured latents, with a CenterPoint-based 3D bounding-box estimator
• Input: Single 360° equirectangular (ERP) panorama
• Output: Complete 3D indoor scene (structural layout + separable, textured asset meshes with PBR materials)
• Base Model: TRELLIS.2-4B (sparse-structure / shape / texture VAEs and decoders)

Key Features

• Structure-aware scene generation: Recovers a coherent global layout from a single 360° image, not just isolated assets.
• Asset-aware output: The scene is decomposed into a layout (floor and walls) and individual objects, each exported as its own mesh rather than a single fused blob.
• View-selective cross-attention: The panorama is unwrapped into 6 cubemap faces (FOV 120°), and each voxel attends only to the faces visible from its 3D position.
• Layout-Guided Structure Inversion (optional): A monocular-depth (Depth-Anything-2) point cloud, the Partial Scene Geometry, seeds coarse generation via SDEdit-style noise inversion for better room-scale fidelity.
• PBR materials: Base color, roughness, metallic, and opacity, inherited from the TRELLIS.2 texture decoder.

Checkpoints

Each folder holds the EMA weight under ckpts/. Model configs ship with the code repository (under configs/), so no config.json is needed here.

Requirements

• System: Tested on Linux.
• Hardware: An NVIDIA GPU with at least 24 GB of memory (verified on NVIDIA A100 and H100).
• Software:
  • The CUDA Toolkit (recommended 12.4).
  • Conda for managing dependencies.
  • Python 3.8 or higher.

Usage

Please refer to the official GitHub repository for installation. InSpace is run through the repository's scripts (demo/app_inspace_*.py, eval/pipeline/eval_pipeline.py), which load these checkpoints and chain the multi-stage pipeline together.

# 1. Get the code and set up the environment (same env as TRELLIS.2)
git clone https://github.com/kookie12/InSpace --recursive && cd InSpace
. ./setup.sh --new-env --basic --flash-attn --nvdiffrast --nvdiffrec --cumesh --o-voxel --flexgemm

# 2. Download the checkpoints into ckpts/ (repo mirrors the local layout)
pip install -U "huggingface_hub[cli]"
hf download GwanHyeong/InSpace --include "ckpts/*" --local-dir .

# 3a. Interactive demo (pick a scene, run the pipeline stage by stage)
python demo/app_inspace_erp_front.py --port 7860

# 3b. Batch inference over the test set
python eval/pipeline/eval_pipeline.py \
    --data_dir datasets/ERP_3D_FRONT_test \
    --noise_mode sdedit --sdedit_alpha 0.5 --bbox_mode predicted

The inference code loads each checkpoint from ckpts/<folder>/ckpts/*.pt; the matching model config is read from the code repository's configs/ directory.

Dataset

InSpace is trained on ERP-FRONT-30K, a paired ERP-Image-to-3D indoor scene dataset built on 3D-FRONT, with 26.5K training and 2.5K test ERP-image-mesh pairs (~30K total). Each room is paired with 360° ERP observations rendered from inside the scene and covers a wide range of room sizes.

hf download GwanHyeong/ERP-FRONT-30K --repo-type dataset --local-dir datasets/

Known Limitations

• Domain of training data: InSpace is trained on ERP-FRONT (synthetic 3D-FRONT scenes). Results on real captured panoramas may vary; for real images the pipeline relies on monocular depth to build the Partial Scene Geometry.
• Raw mesh artifacts: As with TRELLIS.2, generated raw meshes may occasionally contain small holes or minor topological discontinuities; mesh post-processing (hole-filling, remeshing) is provided.

Citation

InSpace has been accepted to ECCV 2026. If you find our work useful, please cite:

@article{koo2026inspace,
  title   = {InSpace: Structure-Aware 3D Indoor Scene Generation from a Single 360{\deg} Image},
  author  = {Koo, Gwanhyeong and Kim, Hyunsu and Kim, Youngji and Lee, Taejae and
             Lim, Siwoo and Yoon, Sunjae and Yeon, Suyong and Yoo, Chang D.},
  journal = {arXiv preprint arXiv:2607.03990},
  year    = {2026}
}

License

Released under the MIT License. This work builds on TRELLIS.2 (MIT, Microsoft). Some dependencies (e.g. nvdiffrast, nvdiffrec) carry their own licenses.