rf-detr-seg-preview-npu

RF-DETR-Seg-Preview-NPU

Run RF-DETR-Seg-Preview on Qualcomm NPU with nexaSDK.

Quickstart

Install nexaSDK and create a free account at sdk.nexa.ai

Activate your device with your access token:

nexa config set license '<access_token>'

Run the model locally in one line:

nexa infer NexaAI/rf-detr-seg-preview-npu

Model Description

RF-DETR-Seg-Preview is a real-time object detection and instance segmentation model developed by Roboflow, based on the Transformer architecture. It is the first real-time model to achieve over 60 AP on the COCO dataset, combining high accuracy with efficient inference performance.

The model leverages a DINOv2 visual backbone and a lightweight DETR design, providing excellent transfer learning capabilities and domain adaptability. Its end-to-end architecture eliminates the need for non-maximum suppression (NMS) and anchor boxes, simplifying the detection pipeline.

RF-DETR-Seg-Preview brings state-of-the-art detection and segmentation accuracy to real-time applications, making it ideal for edge deployment scenarios where both speed and precision are critical.

Features

• High-Accuracy Real-Time Detection & Segmentation: Achieves 60.5 mAP on COCO dataset while maintaining real-time performance, suitable for applications requiring both speed and accuracy.

• Domain Adaptability: Through the DINOv2 backbone network, enables cross-domain transfer learning, suitable for complex scenarios such as aerial imagery and industrial applications.

• Dynamic Resolution Support: Supports multi-resolution training and inference, allowing precision-speed trade-offs without retraining.

• Efficient Edge Deployment: Optimized for Qualcomm NPU, providing fast inference and low latency on edge devices with limited resources.

• End-to-End Architecture: Eliminates the need for NMS and anchor boxes, simplifying the detection and segmentation pipeline.

• Instance Segmentation: Provides pixel-level segmentation masks for each detected object, enabling precise object boundary identification.

Use Cases

• Real-Time Video Analysis: Fast object detection and instance segmentation in image or video streams, suitable for autonomous driving, security monitoring, and surveillance systems.

• Edge Device Deployment: Lightweight design enables deployment on mobile devices, embedded systems, and other edge devices with resource constraints.

• Autonomous Systems: Detection and segmentation of pedestrians, vehicles, and other objects for autonomous navigation and robotics.

• Custom Dataset Fine-Tuning: Supports fine-tuning on custom datasets to meet specific application requirements.

• Production Environments: Efficient deployment in production or research environments requiring real-time performance.

Inputs and Outputs

Input:

• Image path

Output:

• Detection results including object classes, bounding box coordinates, and confidence scores

License

All NPU-related components of this project are licensed under the Creative Commons Attribution–NonCommercial 4.0 International (CC BY-NC 4.0) license.

Commercial licensing or usage rights must be obtained through a separate agreement. For inquiries regarding commercial use, please contact [email protected]