rt-detr-v2-fine-tuned-PPE-Det

RT-DETRv2 Fine-Tuned — PPE Compliance Detection

A fine-tuned RT-DETRv2 (ResNet-50vd backbone) model for real-time Personal Protective Equipment (PPE) compliance monitoring from industrial CCTV footage. Trained for PT Pertamina EP Cepu (PEPC) to detect PPE compliance across 11 classes simultaneously.

⚠️ Critical usage note: This model requires use_fast=False in the image processor (see inference example). The default fast processor in transformers ≥ 5.3 applies incorrect normalization for RT-DETRv2 and will silently produce wrong predictions.

---

Model Details

---

Classes Detected

---

Performance

Evaluated on 964 held-out validation images using COCO evaluation protocol (pycocotools).

Per-class difficulty (qualitative):

• Strong (>80% AP): person, helmet_absent, coverall_present, helmet_present
• Moderate (50–70% AP): coverall_absent, boot_absent, glove_absent
• Weak (<55% AP): boot_present, eyewear_present, eyewear_absent, glove_present

glove_present is the hardest class — only 60 training examples in a 46,200-annotation dataset (133:1 imbalance vs. person).

---

Dataset

Class imbalance:

• Most common: person — 7,984 annotations
• Rarest: glove_present — 60 annotations (133× imbalance)
• Absent classes dominate present classes across all PPE types

Object size distribution:

• Small (<1,024 px²): 38.9%
• Medium (1,024–9,216 px²): 37.1%
• Large (>9,216 px²): 24.0%
• Median bbox area: 1,757 px² — heavy small-object presence

---

Training Configuration

Augmentation pipeline (9 transforms): HorizontalFlip · RandomScale (±15%) · Affine (shift ±5%, rotate ±5°) · RandomCrop · BrightnessContrast · MotionBlur · GaussNoise · CLAHE · RandomShadow → Resize(800×1333)

Class imbalance strategy:

• Weighted random sampling (inverse frequency, cap = 10×)
• Per-class loss weighting

---

Inference

Image inference

from transformers import AutoImageProcessor, AutoModelForObjectDetection
from huggingface_hub import hf_hub_download
import torch
from PIL import Image

# Load processor — use_fast=False is REQUIRED, do not remove
processor = AutoImageProcessor.from_pretrained(
    "PekingU/rtdetr_v2_r50vd",
    use_fast=False
)

# Load base architecture then apply fine-tuned weights
model = AutoModelForObjectDetection.from_pretrained(
    "PekingU/rtdetr_v2_r50vd",
    num_labels=11,
    ignore_mismatched_sizes=True
)
weights_path = hf_hub_download(
    repo_id="abdullahzunorain/rt-detr-v2-fine-tuned-PPE-Det",
    filename="best_model.pt"
)
model.load_state_dict(
    torch.load(weights_path, map_location="cpu", weights_only=True),
    strict=False
)
model.eval()

id2label = {
    0: "person",          1: "coverall_present", 2: "helmet_present",
    3: "helmet_absent",   4: "boot_present",     5: "glove_absent",
    6: "coverall_absent", 7: "eyewear_absent",   8: "boot_absent",
    9: "eyewear_present", 10: "glove_present"
}

image = Image.open("image.jpg").convert("RGB")
inputs = processor(images=image, return_tensors="pt")

with torch.no_grad():
    outputs = model(**inputs)

results = processor.post_process_object_detection(
    outputs,
    threshold=0.3,
    target_sizes=[image.size[::-1]]
)[0]

for score, label, box in zip(results["scores"], results["labels"], results["boxes"]):
    print(f"{id2label[label.item()]}: {score:.2f} @ {[round(c,1) for c in box.tolist()]}")

Video inference (frame-by-frame)

import cv2
from transformers import AutoImageProcessor, AutoModelForObjectDetection
from huggingface_hub import hf_hub_download
import torch
from PIL import Image

processor = AutoImageProcessor.from_pretrained(
    "PekingU/rtdetr_v2_r50vd", use_fast=False
)
model = AutoModelForObjectDetection.from_pretrained(
    "PekingU/rtdetr_v2_r50vd",
    num_labels=11,
    ignore_mismatched_sizes=True
)
weights_path = hf_hub_download(
    repo_id="abdullahzunorain/rt-detr-v2-fine-tuned-PPE-Det",
    filename="best_model.pt"
)
model.load_state_dict(
    torch.load(weights_path, map_location="cpu", weights_only=True),
    strict=False
)
model.eval()

cap = cv2.VideoCapture("video.mp4")
while cap.isOpened():
    ret, frame = cap.read()
    if not ret:
        break
    image = Image.fromarray(cv2.cvtColor(frame, cv2.COLOR_BGR2RGB))
    inputs = processor(images=image, return_tensors="pt")
    with torch.no_grad():
        outputs = model(**inputs)
    results = processor.post_process_object_detection(
        outputs, threshold=0.3, target_sizes=[image.size[::-1]]
    )[0]
    # draw boxes on frame here
cap.release()

---

Limitations

• glove_present recall is low — 60 training examples is insufficient; the model may miss glove compliance in many scenes
• Small object performance — AP_small = 26.4%; gloves and eyewear on distant workers are frequently missed
• Present/absent confusion — the dataset is skewed toward absent (violation) classes, so the model is more reliable at detecting violations than compliance
• Single-site training — trained on one facility's CCTV aesthetic; may not generalize to different lighting conditions, camera angles, or worker attire styles
• 1,532 noisy annotations — bboxes with width or height < 10px exist in training data and introduce noise for small PPE classes
• No segmentation — bbox detection only; not suitable for pixel-level PPE segmentation tasks
• Not for autonomous safety decisions — model predictions require human review before any enforcement action

---

Intended Use

• Industrial PPE compliance monitoring dashboards
• CCTV-based safety auditing systems
• Computer vision research on workplace safety
• Prototyping real-time detection pipelines

Out of scope: Autonomous enforcement, safety-critical decisions without human oversight, or environments significantly different from industrial CCTV settings.

---

Repository Files

---

Citation

@misc{zunorain2026rtdetrv2ppe,
  author = {Abdullah Zunorain},
  title  = {RT-DETRv2 Fine-Tuned PPE Detection},
  year   = {2026},
  url    = {https://huggingface.co/abdullahzunorain/rt-detr-v2-fine-tuned-PPE-Det}
}