LibreRFDETRTinys-visdrone

LibreRFDETRTinys-visdrone: RF-DETR for tiny objects (research preview)

RF-DETR with the TinyFormer small-object modules grafted on (arXiv:2605.25046), trained on VisDrone2019-DET at 640 px. Size s, 44.96 M params, 10 VisDrone aerial classes.

Stock RF-DETR detection models decode from a single stride-16 level. At 640 input that is a 40x40 grid, so a 12-pixel pedestrian gets less than one cell. This variant adds three things:

1. 4-level pyramid (strides 4/8/16/32) instead of one stride-16 level.
2. SSA: a stride-2 conv stem on the raw image. Its stride-8 output fuses into P3; its stride-4 output becomes the F2 decoder level outright (the paper's F2 = SDE2(I)). Detail is re-extracted from pixels, because upsampled ViT features cannot recover what patchification destroyed.
3. PBM: two parallel bi-fusion blocks refining P3 and P4 before the decoder.

Research preview. Trained on VisDrone2019-DET, which is licensed CC BY-NC-SA 3.0, so these weights are for non-commercial use only and are not covered by LibreYOLO's permissive license. The checkpoint detects the 10 VisDrone aerial classes, not COCO.

Results (VisDrone2019-DET val, 548 images, 640x640, pycocotools)

Controlled A/B: both arms warm-started from the same stock RF-DETR COCO checkpoint, identical 60-epoch recipe, best.pt selected by overall mAP in both. The only variable is the architecture.

+1.55 AP_small (+10.3% relative) with the textbook size signature: it wins on small and medium objects and gives back ~2 points on large ones. It reallocates capacity toward fine scales. For aerial/drone imagery that is a good trade; for a general detector it is a real cost.

The curve is the actual evidence. The variant starts 2.1 AP_small behind (it carries 3.45 M randomly-initialised parameters and three feature levels the pretrained decoder has never seen), crosses the baseline around epoch 22, and keeps climbing for 20 more epochs after the baseline has flatlined (the baseline never exceeded 15.22 in 60 epochs). The gain is not "more parameters converge faster" - they converge slower - it is a ceiling the single-level baseline cannot reach at all.

Per-epoch metric history: results.csv.

Honest scope notes

• These numbers are not comparable with published VisDrone results. Our converter drops VisDrone's ignore-regions (1,378 in val, 3.4% of annotations), so detections landing inside them are scored as false positives, whereas the official protocol excludes those areas. Absolute AP here is therefore depressed - the bias runs against us. The TinyFormer paper's own figures (TinyFormer-S-PBM: 18.9 AP_small) also use a DINOv3 backbone with Objects365 pretraining, against our DINOv2 + COCO warm start. Do not read this as beating or matching them. The defensible claim is the A/B: SSA+PBM beats an identically-trained stock RF-DETR.
• Single run, single seed. No error bars. The +1.55 effect is well outside the +/-0.3 checkpoint-to-checkpoint wobble, but a seed sweep would be stronger.
• The +40% parameter cost is mostly avoidable waste. The SSA/PBM modules are only 3.45 M of it; 9.19 M is RF-DETR's MultiScaleProjector, which builds each pyramid scale with a full C2f over four ViT features. The paper's cheap SPB gets the same pyramid for ~2 M, which would cut overhead to +13%. A slim v2 is planned; prefer it when it lands.
• Match the architecture to the arena. This helps where objects are 4-16 px. On COCO, where "small" is 16-32 px and already covered by a stride-8 level, the extra machinery is unlikely to pay for itself.

Usage

The rfdetr_so family is not in the LibreYOLO release, so this checkpoint does not auto-download. Get the model code from the experiment repo:

git clone https://github.com/LibreYOLO/rf-detr-tinyobjects-v1
pip install "libreyolo[rfdetr]"

from librfdetr_tiny import LibreRFDETRTiny

model = LibreRFDETRTiny("LibreRFDETRTinys-visdrone.pt")
results = model.predict("drone.jpg", imgsz=640)

Classes (VisDrone 1..10, remapped to 0..9): pedestrian, people, bicycle, car, van, truck, tricycle, awning-tricycle, bus, motor.

Reproduce

python build_visdrone.py --datasets <datasets>      # official zips -> YOLO format
python train_so.py --data <datasets>/visdrone/visdrone.yaml

Full experiment, plans, per-epoch logs, the A/B chart and the threats to validity: LibreYOLO/rf-detr-tinyobjects-v1.

Source and provenance

• Architecture: LibreYOLO's port of roboflow/rf-detr (Apache-2.0), itself derived from LW-DETR, Conditional DETR, DETR and ViTDet. Backbone is DINOv2 (Apache-2.0).
• SSA and PBM were implemented clean-room from the method description in the TinyFormer paper (arXiv:2605.25046). The paper's linked code repository returned 404 at implementation time (2026-07-11) and was never consulted.
• Training data: VisDrone2019-DET (AISKYEYE team, Tianjin University), CC BY-NC-SA 3.0. Images are not redistributed here; build_visdrone.py is a clean-room converter written from the public annotation spec.

License

CC BY-NC-SA 3.0 (inherited from the VisDrone training data): non-commercial use only. The model code carries its own licenses (RF-DETR lineage is Apache-2.0); see the source repositories.