RF-DETR-Large-detection-ADOPD

RF-DETR Large Detection ADOPD

Thinking with Anchors Project | ADOPD 2026 Paper: Thinking with Anchors: Grounded and Efficient Document Reasoning | ADOPD 2024 Paper | Dataset | Code

Use Restrictions

Non-commercial research use only. The ADOPD fine-tuned checkpoint weights in this repository are provided solely for non-commercial research. Commercial use of these checkpoint weights is not permitted. Users must also comply with every applicable upstream license and acceptable-use term; see USE_RESTRICTIONS.md.

Model Overview

• Model developer: Thinking with Anchors project contributors
• Base architecture: RF-DETR Large
• Model type: transformer object detector
• Task: grouped document text-region detection (Doc2Box)
• Fine-tuning dataset: adopd/adopd2026
• Input: RGB document image
• Output: text-region boxes, confidence scores, and class IDs

Description

RF-DETR Large Detection ADOPD is an RF-DETR Large model fine-tuned for single-class grouped text-region detection in document images. It predicts the spatial extent of text groups rather than individual characters or OCR transcriptions.

Training Data

This checkpoint is fine-tuned on the ADOPD Doc2Box task. Public supervision is stored in:

ocr.grouped_blocks[].bbox_xyxy

The companion exporter converts this field to a one-class COCO dataset with the class name text.

Checkpoint Format

model.ckpt is a PyTorch Lightning checkpoint for the RF-DETR large architecture. It is not the separate RFDETR.from_checkpoint() export format. Use the pinned RF-DETR revision and the ADOPD loader shown below.

Quick Start

1. Install

git clone https://github.com/SichenZhu/ADOPD2026.git
cd ADOPD2026/release_code

git clone https://github.com/roboflow/rf-detr.git upstream/rf-detr
git -C upstream/rf-detr checkout 7f2490d4ece5a894b6bfe69e876a1d5d9936e2e1

python -m pip install -e model_zoo/common
python -m pip install -e 'upstream/rf-detr[train,loggers]'
python -m pip install -e model_zoo/rf_detr

2. Download And Run

hf download adopd/RF-DETR-Large-detection-ADOPD \
  --local-dir checkpoints/rfdetr-text

adopd-rfdetr-infer \
  --checkpoint checkpoints/rfdetr-text/model.ckpt \
  --architecture large \
  --image document.jpg \
  --threshold 0.5 \
  --output prediction.json

The output JSON contains boxes_xyxy, scores, class_ids, and image size.

Fine-Tuning And Evaluation

Prepare train and validation with adopd-rfdetr-prepare --task detect, then train with adopd-rfdetr-train --architecture large --task detect. Full commands are documented in rf_detr.

Limitations

This model detects grouped text regions but does not perform text recognition. It is single-class and may require confidence-threshold calibration for new document domains.

License

The ADOPD fine-tuned checkpoint weights are subject to the non-commercial, research-only restriction above. The included Apache License 2.0 documents the terms applicable to upstream RF-DETR software components; it does not replace the checkpoint-weight restriction. Use is permitted only when all applicable terms are satisfied.

Citation

Please cite the ADOPD 2026 and ADOPD 2024 papers.

@misc{zhu2026thinkingwithanchors,
  title={Thinking with Anchors: Grounded and Efficient Document Reasoning},
  author={Sichen Zhu and Yuchen Zhu and Wenzhuo Xu and Jason Kuen and Wanrong Zhu and Jing Shi and Xuan Shen and Quanyi Wang and Yiwei Wang and Yujun Cai and Bing Shuai and Qin Zhang and Yongxin Chen and Shilong Liu and Molei Tao and Jiuxiang Gu},
  year={2026}
}

@inproceedings{gu2024adopd,
  title={{ADOPD}: A Large-Scale Document Page Decomposition Dataset},
  author={Jiuxiang Gu and Xiangxi Shi and Jason Kuen and Lu Qi and Ruiyi Zhang and Anqi Liu and Ani Nenkova and Tong Sun},
  booktitle={The Twelfth International Conference on Learning Representations},
  year={2024},
  url={https://openreview.net/forum?id=x1ptaXpOYa}
}