rfdetr-urban-segmentation

Model Description

This model is an RF-DETR instance segmentation network specifically trained to detect and segment urban sidewalks from monocular street-level imagery (e.g., Google Street View).

It serves as the foundational feature extractor for the Alivestreets spatial metric pipeline, where the generated pixel masks are geometrically transformed into real-world physical dimensions (meters) using ground plane homography, Oriented Bounding Boxes (OBB), and Dynamic Perspective Scale Compensation.

Intended Use

• Urban infrastructure mapping and accessibility analysis.
• Extraction of navigable pedestrian areas from street-level images.
• Integration into the yolo-pipeline and ground-truth-sidewalk repositories for physical width estimation.

Out-of-Scope Use

• The model is not designed for depth estimation by itself (it outputs 2D pixel masks).
• Performance degrades significantly in non-urban environments or extreme topographical slopes.

Training Data

The model was trained on a custom dataset managed via Roboflow, containing annotated instances of urban sidewalks.

• Source: cfcs-workspace/sidewalks-epofw on Roboflow Universe.

Evaluation Results

The model was evaluated using the standard COCO mAP (Mean Average Precision) metric via the supervision library. It demonstrates highly robust spatial reliability for medium and large masks (which typically correspond to the main navigable sidewalk).

Limitations and Bias

• Texture Bias: The training dataset predominantly features standard concrete sidewalks. The model may exhibit false negatives or fragmented masks when exposed to anomalous textures (e.g., cobblestone, painted tiles, dirt, or grass-covered pathways).
• Semantic Erosion: Deep learning downsampling and pooling operations tend to slightly erode the semantic boundaries of the mask. In our metric estimation pipeline, this is compensated deterministically with a +0.1m physical calibration factor.
• Small Areas: As seen in the metrics (AP small = 0.123), the model struggles with very distant or heavily occluded sidewalk fragments.

How to Get Started with the Model

You can use the model within the Alivestreets inference pipeline. Example usage requiring the custom rfdetr wrapper:

import cv2
from PIL import Image
from rfdetr import RFDETRSegSmall

# 1. Load Model
model_path = "path/to/downloaded/sidewalk_rfdetr.pth"
model = RFDETRSegSmall(pretrain_weights=model_path)
model.optimize_for_inference()

# 2. Predict
image = cv2.imread("street_view.jpg")
image_rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
pil_image = Image.fromarray(image_rgb)

detections = model.predict(pil_image, threshold=0.5)
print(detections)