faster-rcnn-zod

Model Card: Faster R-CNN R101-FPN on ZOD (Vehicle, Pedestrian, VulnerableVehicle)

Model Details

• Architecture: Faster R-CNN with ResNet-101 + FPN (Detectron2 model zoo config COCO-Detection/faster_rcnn_R_101_FPN_3x.yaml)
• Framework: Detectron2

Training Data

• Source: Zenseact Open Dataset (ZOD), converted to COCO
• Classes kept: Vehicle, Pedestrian, VulnerableVehicle
• Filtering: exclude occlusion levels Heavy and VeryHeavy (keep None/Medium/Light); drop boxes with height < 25 px
• Split: original ZOD train is split into train/val by 1 km lat/lon tiles (80/20; 69,090 train / 20,882 val) to reduce spatial leakage and keep weather/road_type/time_of_day distributions similar; original ZOD val is used as test
• Anonymization: used the conversion script default (BLUR)

Training Procedure

• Augmentations: random horizontal flip, brightness/contrast/saturation jitter, ResizeShortestEdge
• Solver: IMS_PER_BATCH=16, BASE_LR=0.001, MAX_ITER=150000, STEPS=(80000, 110000), WARMUP_ITERS=1000
• Evaluation: COCO evaluator every 1000 iterations

Evaluation Results

Metrics (iteration 104999, on my defined val split):

• AP: 34.2
• AP50: 59.4
• AP75: 33.3
• AP-Pedestrian: 23.2
• AP-Vehicle: 55.2
• AP-VulnerableVehicle: 24.2
• APl: 53.3
• APm: 31.4
• APs: 10.8

Usage (Detectron2)

import cv2
from detectron2.config import get_cfg
from detectron2.engine import DefaultPredictor
from detectron2.utils.visualizer import Visualizer
from detectron2.data import MetadataCatalog
cfg = get_cfg()
cfg.merge_from_file("config.yaml")
cfg.MODEL.WEIGHTS = "model_final.pth"
cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.5
# cfg.MODEL.DEVICE = "cpu"  # uncomment for CPU
predictor = DefaultPredictor(cfg)
image = cv2.imread("path/to/image.jpg")
outputs = predictor(image)
# Optional visualization
metadata = MetadataCatalog.get("zod_infer")
metadata.thing_classes = ["Vehicle", "Pedestrian", "VulnerableVehicle"]
vis = Visualizer(image[:, :, ::-1], metadata=metadata, scale=1.0)
vis = vis.draw_instance_predictions(outputs["instances"].to("cpu"))
cv2.imwrite("predictions.jpg", vis.get_image()[:, :, ::-1])

Limitations

• Trained on only three classes; other ZOD object types are not represented.
• Heavy/VeryHeavy occlusions and very small objects (<25 px height) are filtered out and may perform poorly at inference time.