sitesense-weights

🏗️ SiteSense — Model Weights

Real-Time Construction Equipment Monitoring via Aerial Computer Vision

---

Overview

This repository hosts the trained model weights for SiteSense — a real-time pipeline that detects, tracks, identifies, and classifies the activity of heavy construction equipment from drone/aerial video footage.

The system processes each frame through a multi-phase pipeline:

Video Frame → Detector (RF-DETR or YOLO26-L) → BoT-SORT Tracking → DINOv3 Re-ID → Activity Classification → Kafka Events

Two interchangeable detectors are provided. Switch at runtime via the DETECTOR_TYPE environment variable (rfdetr or yolo) — no rebuild required.

---

Model Weights

Note: The DINOv3 ViT-B/16 backbone (~327 MB) is not included here. It is auto-downloaded from facebook/dinov3-vitb16-pretrain-lvd1689m on first run using your HF_TOKEN.

---

Detection Classes

Both detectors are fine-tuned on the same merged MOCS + ACID v2 dataset to recognize 8 classes of construction equipment from aerial perspectives:

---

Training Results

Both detectors were trained on the identical train/val/test split (42,733 / 4,615 / 990 images) for direct comparison. Numbers below are on the held-out val split.

Detector Comparison (val split)

RF-DETR wins on 7 of 8 per-class AP50-95 (only bulldozer goes to YOLO26-L: 0.796 vs 0.785). The largest RF-DETR margins are on the most under-represented classes — mobile_crane (+4.7 pts) and tower_crane (+6.0 pts) — where set-based prediction handles long boom shapes and heavy occlusion better than YOLO's anchor-based head.

<details> <summary><strong>Per-class AP@50:95</strong></summary> </details>

DINOv3 Re-ID Projection Head

---

Quick Start

Option A: Download All Weights (Recommended)

pip install huggingface_hub
huggingface-cli download Zaafan/sitesense-weights --local-dir models/

This pulls all four weight files at once into your models/ directory — both detectors plus both Re-ID heads.

Option B: Python API

from huggingface_hub import hf_hub_download

# Detectors (pick one or both)
hf_hub_download(repo_id="Zaafan/sitesense-weights", filename="rfdetr_construction.pth",     local_dir="models/")
hf_hub_download(repo_id="Zaafan/sitesense-weights", filename="yolo26l_construction_v1.pt",  local_dir="models/")

# Re-ID
hf_hub_download(repo_id="Zaafan/sitesense-weights", filename="dinov3_reid_head.pth",        local_dir="models/")
hf_hub_download(repo_id="Zaafan/sitesense-weights", filename="osnet_x0_25_msmt17.pt",       local_dir="models/")

Option C: Auto-Download (Zero Setup)

The SiteSense pipeline automatically downloads missing weights on first run:

# In services/cv-inference/main.py — resolve_weights() handles this transparently.
# It picks the right file based on DETECTOR_TYPE (yolo or rfdetr).
weights_path = resolve_weights('yolo26l_construction_v1.pt')  # local first, HF fallback

---

Usage with SiteSense Pipeline

# 1. Clone the repository
git clone https://github.com/Mahmoud-Zaafan/SiteSense.git
cd SiteSense

# 2. Download weights
huggingface-cli download Zaafan/sitesense-weights --local-dir models/

# 3. Configure environment
cp .env.example .env

# 4. Launch infrastructure
docker compose up --build -d

# 5a. Run pipeline with the default detector (YOLO26-L)
docker compose --profile pipeline up cv-inference

# 5b. Or switch to RF-DETR at runtime — no rebuild needed
DETECTOR_TYPE=rfdetr docker compose --profile pipeline up cv-inference

---

Citation

If you use these weights in your research or projects, please cite:

@misc{sitesense2025,
  author = {Mahmoud Zaafan},
  title  = {SiteSense: Real-Time Construction Equipment Monitoring via Aerial Computer Vision},
  year   = {2025},
  url    = {https://github.com/Mahmoud-Zaafan/SiteSense}
}

---

License

All weights are released under the MIT License.