pxwplanar-moge2-planarity

Pixel-wise Planarity — 4-head MoGe-2 planarity model

Model weights for Pixel-wise Planarity for High-Precision Monocular Plane Segmentation (ECCV 2026). A MoGe-2 ViT-L backbone fine-tuned with a fourth head that predicts per-pixel planarity probability, alongside MoGe-2's metric depth, surface normals, and validity mask. Combined with GPU-accelerated region growing, the four signals yield a plane segmentation from a single RGB image.

Code, demo, benchmark, and ground-truth generation pipeline: https://github.com/alpayozkan/PixelwisePlanarity

Usage

# pip install -e .  (see the GitHub repo; requires the MoGe submodule)
from pxwplanar.inference.planarity.moge_inference import MoGePlanarityInference
from pxwplanar.shared.segmentation import compute_planar_segments
import numpy as np

model = MoGePlanarityInference.from_pretrained("alpayozkan/pxwplanar-moge2-planarity")
res = model.predict_metric("image.jpg", num_tokens=1600, return_all_heads=True)
# res: planarity_probability, depth (m), normal, points, mask, intrinsics

labels, n = compute_planar_segments(
    (res["planarity_probability"] > 0.3).astype(np.int16),
    res["normal"], res["depth"],
    np.deg2rad(5.0), 0.025, neighbor_match_count_thresh=8)

The checkpoint is stored in MoGe's native from_pretrained format ({'model_config', 'model'}), so it also loads directly through the MoGe fork bundled with the repo:

from MoGe.moge.model.v2 import MoGeModel
model = MoGeModel.from_pretrained("alpayozkan/pxwplanar-moge2-planarity")

Model details

Canonical segmentation parameters

Used identically in the demo, inference scripts, and benchmark: planarity > 0.3, normal threshold 5.0°, relative depth threshold 0.025, ≥8 matching neighbors.

Evaluation

Benchmarked on ScanNet++ (indoor) and SYNTHIA / VKITTI2 (outdoor) against rendered plane GT: 2D segmentation (Segmentation Covering, Rand Index, Variation of Information), 3D precision/recall at 1/5/10 mm (RANSAC-fitted planes vs GT), and binary-planarity accuracy/precision/recall/F1/IoU. See the GitHub repo's evaluation suite (evaluate_all_baselines.py) to reproduce.

License

MIT. This checkpoint is a fine-tune of Microsoft's MoGe-2 weights (MIT); the encoder is DINOv2 (Meta AI, Apache 2.0) as redistributed within MoGe. See the LICENSE file in this repo for the retained notices; model files were modified relative to the originals (added planarity head, fine-tuned weights).

Citation

@inproceedings{pixelwiseplanarity2026,
  title     = {Pixel-wise Planarity for High-Precision Monocular Plane Segmentation},
  author    = {Yavuz, Ahmetcan and Ozkan, Alpay and Pautrat, R{\'e}mi and Liu, Shaohui and Pollefeys, Marc},
  booktitle = {Proceedings of the European Conference on Computer Vision (ECCV)},
  year      = {2026}
}