reverse_qwen25_vl

REVERSE-Qwen2.5-VL-3B

<a href="https://arxiv.org/abs/2504.13169"> <img src="https://img.shields.io/badge/arXiv-2504.13169-b31b1b.svg" alt="arXiv" /> </a>

Model Summary

REVERSE-Qwen2.5-VL-3B is a novel open-source vision-language model (VLM) that performs both next-token predictioin and self-verification / self-correction during the generation process. Built on top of Qwen2.5-VL-3B-Instruct, it is fine-tuned using the 100k-subset of REVERSE Visual Instruct 1.3M dataset and equipped with a retrospective resampling mechanism that allows it to detect and correct hallucinations during generation. The model is trained in early May, 2025.

Performance

REVERSE achieves state-of-the-art hallucination reduction across diverse captioning and open-ended visual question answering benchmarks. To ensure the apple-to-apple comparison, we fine-tune the released Qwen2.5-VL-3B model using both the LLaVA-FT setup and our REVERSE recipe, applying both on the same 100k subset. This allows us to directly compare the impact of our method against the LLaVA-FT baseline under consistent conditions as the Qwen2.5-VL's instruction tuning data is not publicly available.

It also performs competitively on discriminative tasks compared with the base VLM.

Usage

Please refer to the installation guide on GitHub to get started:
👉 Installation Guide

Additional Resources

• 📄 Project Page: https://reverse-vlm.github.io/
• 🧾 Dataset: REVERSE Visual Instruct 1.3M
• 🔧 Ask Questions: GitHub Issues

Intended Use

Primary Use Cases:

• Reducing hallucination in image captioning and VQA tasks
• Benchmarking hallucination-aware generation
• Research on grounded vision-language generation and self-correction

Target Users:
Researchers, developers, and students working in computer vision, NLP, and multimodal AI.