doorway-door-people-window-seg

Doorway / Door / People / Window real-time segmenter (YOLO26n-seg)

A tiny (3.1 M-param) instance-segmentation model for indoor navigation, distilled from SAM 3 (offline concept-prompt teacher) into a Jetson-deployable YOLO26n-seg student.

Classes (output)

Class 4 = mirror is internal: it is detected but never output — used at inference to suppress door/doorway/window predictions that fall on a mirror (reflections are phantom obstacles). Run predictions through mirror_suppress.finalize().

Key properties (measured on held-out slices)

• No double-detection on open doors: 0/78 (0.0%) — a door and doorway never both fire on one portal (enforced by a labeling-time partition + an inference-time portal de-dup).
• Closed glass door → door: ~79–82% (conference-room case).
• Mirror reflection false-positives: ~6/77 frames (down from 8/77 without the mirror class).
• Overall (combined val, incl. hard OOD corridor/university glass): box mAP50 0.52, mask mAP50 0.48.

Files

• doorway_seg_yolo26n.pt — Ultralytics weights (train/val/predict).
• doorway_seg_yolo26n.onnx — NMS-free ONNX (opset 17, 640×640) for TensorRT/ONNXRuntime; build the Jetson engine on-device with trtexec.
• mirror_suppress.py — inference post-process: mirror suppression + portal de-dup (finalize).
• predict.py — demo: run on image/video/webcam with the post-process applied.

Usage

from ultralytics import YOLO
from mirror_suppress import finalize, dets_from_result   # from this repo

model = YOLO("doorway_seg_yolo26n.pt")
r = model.predict("frame.jpg", conf=0.10, imgsz=640)[0]   # predict low; finalize re-thresholds
dets = finalize(dets_from_result(r), out_conf=0.35)        # 4-class output, reflections dropped
# dets: [{"cls":0..3, "box":[x1,y1,x2,y2], "conf":...}, ...]

Training data (auto-labeled by SAM 3, no human masks)

ADE20K, DoorDetect, COCO-person (capped), Trans10K-v2 (glass door/window), + active-learning from office/corridor/industrial and university walkthrough videos. ADE20K ground-truth mirror masks provide the internal mirror class. Quantization: FP16 lossless; INT8 PTQ ≈ −3 mAP pts.

Limitations

• doorway is the rarest/hardest class (small AP); open passages viewed straight-on can read as door.
• Mirror suppression is limited by mirror-detection recall (~65–70%); some reflected windows remain.
• Domain-sensitive confidence — fine-tune on your deployment footage (active-learning loop) for best results.

Distilled with SAM 3 (Meta). Student: Ultralytics YOLO26n-seg.