Visurf-7B-Best-on-gRefCOCO

ViSurf: Visual Supervised-and-Reinforcement Fine-Tuning for Large Vision-and-Language Models

This repository contains the model presented in the paper ViSurf: Visual Supervised-and-Reinforcement Fine-Tuning for Large Vision-and-Language Models.

GitHub Repository: https://github.com/dvlab-research/ViSurf

Abstract

Typical post-training paradigms for Large Vision-and-Language Models (LVLMs) include Supervised Fine-Tuning (SFT) and Reinforcement Learning with Verifiable Rewards (RLVR). SFT leverages external guidance to inject new knowledge, whereas RLVR utilizes internal reinforcement to enhance reasoning capabilities and overall performance. However, our analysis reveals that SFT often leads to sub-optimal performance, while RLVR struggles with tasks that exceed the model's internal knowledge base. To address these limitations, we propose ViSurf (\textbf{Vi}sual \textbf{Su}pervised-and-\textbf{R}einforcement \textbf{F}ine-Tuning), a unified post-training paradigm that integrates the strengths of both SFT and RLVR within a single stage. We analyze the derivation of the SFT and RLVR objectives to establish the ViSurf objective, providing a unified perspective on these two paradigms. The core of ViSurf involves injecting ground-truth labels into the RLVR rollouts, thereby providing simultaneous external supervision and internal reinforcement. Furthermore, we introduce three novel reward control strategies to stabilize and optimize the training process. Extensive experiments across several diverse benchmarks demonstrate the effectiveness of ViSurf, outperforming both individual SFT, RLVR, and two-stage SFT \textrightarrow RLVR. In-depth analysis corroborates these findings, validating the derivation and design principles of ViSurf.

Overview of ViSurf

<div align=center> <img width="98%" src="https://github.com/dvlab-research/ViSurf/raw/main/assets/overview.png"/> </div>

ViSurf (Visual Supervised-and-Reinforcement Fine-Tuning) is a unified post-training paradigm that integrates the strengths of both SFT and RLVR within a single stage.

Basic Usage with Transformers

This section demonstrates how to load the model using the Hugging Face transformers library.

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_name = "Ricky06662/Visurf-7B-Best-on-gRefCOCO"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.float16)

News

[Oct. 12th, 2025] 🔥 ViSurf is coming! We have released the code and training data.

Contents

• Installation
• Inference
• Evaluation
• Training
• Build Your Data
• Citation
• Acknowledgement

Installation

git clone https://github.com/dvlab-research/ViSurf.git
cd ViSurf
conda create -n visionreasoner python=3.12
conda activate visionreasoner
pip install -e .

Inference

Download pretrained models using the following scripts:

mkdir pretrained_models
cd pretrained_models
git lfs install
git clone https://huggingface.co/Ricky06662/Visurf-7B-Best-on-gRefCOCO

[!TIP] If you encounter issues with connecting to Hugging Face, consider using export HF_ENDPOINT=https://hf-mirror.com.

Then run inference using:

python inference_scripts/inference_visurf.py

The default question is

"I want to rest, where should I sit?"

You will get the thinking process in command line, like:

"The question seems to be asking where to sit, but the image only shows a kitchen counter with food and flowers."

And the mask will be presented in inference_scripts folder. In this case, there is no related object.

<div align=center> <img width="98%" src="https://github.com/dvlab-research/ViSurf/raw/main/assets/test_output_1.png"/> </div>

You can also try find objects in the image by:

python inference_scripts/inference_visurf.py --text "I want to cook food, what can I use?"

You will get the thinking process in command line, like:

"The question asks what kitchen tools or ingredients are visible that could be used for cooking."

The mask will be presented in inference_scripts folder.

<div align=center> <img width="98%" src="https://github.com/dvlab-research/ViSurf/raw/main/assets/test_output_2.png"/> </div>

You can also provide your own image_path and text by:

python inference_scripts/inference_visurf.py --image_path "your_image_path" --text "your question text"

Evaluation

Evaluation Data: 🤗 gRefCOCO val

We recommend you to VisionReasoner for evaluation on ViSurf.

[!NOTE] In ViSurf, the best results on different benchmark are evaluated using different checkpoint. We only release best ckpt on gRefCOCO. For someone who may care about the performance, we suggest you can evaluate and compare the value in your environment.

Training

1. ViSurf Training

Training Data: 🤗 ViSurf 7300 Download dataset using this script:

python training_scripts/download_dataset.py

[!TIP] Try resize the image and re-calculate the corresponding bbox/point coordinates if you have lower GPU memory. Remember changing the corresponding resize_size in evaluation and inference.

Download pretrained models using the following scripts:

mkdir pretrained_models
cd pretrained_models
git lfs install
git clone https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct

(Optional) Start Ray in advance by:

ray start --head  # or ray start --head -- port xxxx

Start training using this script:

bash training_scripts/qwen2_5vl_visurf_nonobj_7300.sh

You can try change the following hyper-parameters if you have a large GPU memory.

worker.actor.micro_batch_size_per_device_for_update=1 or 2 or 4 or 8 or 16 \
worker.actor.micro_batch_size_per_device_for_experience=1 or2 or 4 or 8 or 16 \

If your GPU has less memory, you can change the following config. The number is depend on your GPU memory.

worker.rollout.tensor_parallel_size=[your number between 1-4]
worker.rollout.gpu_memory_utilization=[your number between 0-1]
worker.rollout.n=[your number between 2-32]

2. Merge Checkpoint in Hugging Face Format

python3 training_scripts/model_merger.py --local_dir [path_to_your_actor_checkpoint]

Build Your Own Training Data (Optional)

Please refer to SegZero if you want to build your own dataset.

Citation

@article{liu2025visurf,
  title        = {ViSurf: Visual Supervised-and-Reinforcement Fine-Tuning for Large Vision-and-Language Models},
  author       = {Liu, Yuqi and Chen, Liangyu and Liu, Jiazhen and Zhu, Mingkang and Zhong, Zhisheng and Yu, Bei and Jia, Jiaya},
  journal      = {arXiv preprint arXiv:2503.06520},
  year         = {2025}
}


@article{liu2025segzero,
  title        = {Seg-Zero: Reasoning-Chain Guided  Segmentation via Cognitive Reinforcement},
  author       = {Liu, Yuqi and Peng, Bohao and Zhong, Zhisheng and Yue, Zihao and Lu, Fanbin and Yu, Bei and Jia, Jiaya},
  journal      = {arXiv preprint arXiv:2503.06520},
  year         = {2025}
}

@article{liu2025visionreasoner,
  title        = {VisionReasoner: Unified Visual Perception and Reasoning via Reinforcement Learning},
  author       = {Liu, Yuqi and Qu, Tianyuan and Zhong, Zhisheng and Peng, Bohao and Liu, Shu and Yu, Bei and Jia, Jiaya},
  journal = {arXiv preprint arXiv:2505.12081},
  year         = {2025}
}

Acknowledgement

We would like to thank the following repos for their great work:

• This work is built upon the EasyR1 and veRL.
• This work utilizes models from Qwen2-VL, Qwen2.5-VL and SAM2.

Star History