project_sif

SIF — Detail-Preserving 3D Semantic Segmentation

Part of the ANIMA Perception Suite by Robot Flow Labs.

Paper

Details Matter: Fine-Grained Indoor 3D Segmentation with Open-Vocabulary (Jul 2025) ANIMA Research Institute

Architecture

DetailAwarePointNet — a PointNet++ variant with detail-preserving set abstraction layers designed for fine-grained indoor 3D semantic segmentation. Preserves small objects, thin structures, and subtle boundaries that coarse 3D methods miss.

• Parameters: 705,979
• Input: Points (B, N, 3) + Features (B, N, 6) [XYZ + RGB/normals]
• Output: Logits (B, N, 59) + Features (B, N, 512)
• Backbone: 4-level encoder (SA1–SA4) with detail-preserving radii + feature propagation decoder with skip connections

Training

Exported Formats

Note: TensorRT .trt engine files are hardware-specific and should be built on the target device from the TRT-optimized ONNX.

Additional Files

Quick Start

import torch
from safetensors.torch import load_file
from anima_sif.backbone.detail_aware_pointnet import DetailAwarePointNet

# Load model
state = load_file("pytorch/sif_v1.safetensors")
model = DetailAwarePointNet(in_dim=6, num_classes=59, feature_dim=512)
model.load_state_dict(state)
model.eval()

# Inference
points = torch.randn(1, 8192, 3)    # XYZ coordinates
features = torch.randn(1, 8192, 6)  # XYZ + RGB or normals
out = model(points, features)
# out["logits"]: (1, 8192, 59)
# out["features"]: (1, 8192, 512)

ONNX Inference

import onnxruntime as ort
import numpy as np

sess = ort.InferenceSession("onnx/sif_v1.onnx")
points = np.random.randn(1, 8192, 3).astype(np.float32)
features = np.random.randn(1, 8192, 6).astype(np.float32)
logits, feats = sess.run(None, {"points": points, "features": features})

ROS2 Interface

Docker

docker pull ghcr.io/robotflow-labs/anima-sif:0.1.0
docker run --gpus all -p 8080:8080 ghcr.io/robotflow-labs/anima-sif:0.1.0

License

Apache 2.0 — Robot Flow Labs / AIFLOW LABS LIMITED

Built with ANIMA by Robot Flow Labs.