LocateAnything-3B-dual-image-tagger-ADOPD

LocateAnything-3B Dual-Image Tagger ADOPD

Thinking with Anchors Project | ADOPD 2026 Paper: Thinking with Anchors: Grounded and Efficient Document Reasoning | ADOPD 2024 Paper | Dataset | Code

Use Restrictions

Non-commercial research use only. The ADOPD fine-tuned checkpoint weights in this repository are provided solely for non-commercial research. Commercial use of these checkpoint weights is not permitted. Users must also comply with every applicable upstream license and acceptable-use term; see USE_RESTRICTIONS.md.

Model Overview

• Model developer: Thinking with Anchors project contributors
• Base model: nvidia/LocateAnything-3B
• Model type: dual-image vision-language region classifier
• Task: document entity tagging (Doc2Tag)
• Fine-tuning dataset: ADOPD region masks and semantic tags
• Input: marked full-page image, region crop, and closed-set prompt
• Output: one class in <ref>CLASS</ref> format

Description

This model classifies one supplied region in a document image. The companion pipeline creates two views from the source image and target geometry:

1. the full document page with the target region highlighted in red; and
2. a close-up crop of the same region.

Both images and the closed-set class list are sent to LocateAnything in one request. The model returns a single region class.

Training Data And Label Space

The checkpoint was fine-tuned on its 12-class Doc2Tag label space. The current ADOPD2026 release stores updated region labels under:

human_annotated_masks[].vlm_annotation.label

The current public taxonomy contains 12 target labels plus Other / Not Target and is not identical to the checkpoint's training label space. Use --taxonomy legacy for this checkpoint. Fine-tune on the public recipe before using the updated taxonomy as the output space.

Quick Start

git clone https://github.com/SichenZhu/ADOPD2026.git
cd ADOPD2026/release_code

python -m pip install -e model_zoo/common
python -m pip install -e model_zoo/locateanything_3b_tagger

hf download adopd/LocateAnything-3B-dual-image-tagger-ADOPD \
  --local-dir checkpoints/locany-tagger

adopd-locany-tagger-infer \
  --checkpoint checkpoints/locany-tagger \
  --taxonomy legacy \
  --image document.jpg \
  --bbox X0,Y0,X1,Y1 \
  --output prediction.json

The output contains the parsed tag and complete raw model response. Inference requires a CUDA-capable NVIDIA GPU. Eagle source and its compatibility overlay are only required for fine-tuning.

Fine-Tuning And Evaluation

Use adopd-locany-tagger-prepare to build a current-taxonomy recipe from ADOPD2026. Complete training and evaluation commands are in locateanything_3b_tagger.

Limitations

This model classifies a supplied region and does not discover regions. Its training label space differs from the updated public taxonomy. Marking and crop geometry can affect predictions.

License

The ADOPD fine-tuned checkpoint weights are subject to the non-commercial, research-only restriction above. The included NVIDIA License and the terms for the LocateAnything base model and bundled remote-code files also apply. Use is permitted only when all applicable terms are satisfied.

Citation

Please cite the ADOPD 2026 and ADOPD 2024 papers.

@misc{zhu2026thinkingwithanchors,
  title={Thinking with Anchors: Grounded and Efficient Document Reasoning},
  author={Sichen Zhu and Yuchen Zhu and Wenzhuo Xu and Jason Kuen and Wanrong Zhu and Jing Shi and Xuan Shen and Quanyi Wang and Yiwei Wang and Yujun Cai and Bing Shuai and Qin Zhang and Yongxin Chen and Shilong Liu and Molei Tao and Jiuxiang Gu},
  year={2026}
}

@inproceedings{gu2024adopd,
  title={{ADOPD}: A Large-Scale Document Page Decomposition Dataset},
  author={Jiuxiang Gu and Xiangxi Shi and Jason Kuen and Lu Qi and Ruiyi Zhang and Anqi Liu and Ani Nenkova and Tong Sun},
  booktitle={The Twelfth International Conference on Learning Representations},
  year={2024},
  url={https://openreview.net/forum?id=x1ptaXpOYa}
}