Qwen2.5-VL-3B-GRPO-Thinking

Model Card: Qwen2.5-VL-3B-GRPO-Thinking

The Qwen2.5-VL-3B-GRPO-Thinking is a multi-modal Vision-Language Model (VLM) capable of processing both images and text. This version is fine-tuned from the foundational Qwen2.5-VL-3B using the GRPO (Group Relative Policy Optimization) reinforcement learning algorithm. The primary goal of this fine-tuning is to equip the model with enhanced logical reasoning and a step-by-step "thinking process" to solve complex tasks more accurately before providing the final answer.

Model Details

Model Description

This model retains the excellent visual perception capabilities of the Qwen2.5-VL ecosystem (including object detection, OCR, chart analysis, and dense image understanding). The key differentiator is that it has been trained to prioritize generating intermediate analytical steps (Chain-of-Thought) before formulating the final response.

• Developed by: Manh1011
• Model type: Multi-modal Large Language Model (Causal LM)
• Language(s): Multilingual, highly optimized for English and Vietnamese.
• License: Apache-2.0 (inherited from the base model).
• Finetuned from model: Qwen/Qwen2.5-VL-3B-Instruct

Model Sources

• Repository: Manh1011/Qwen2.5-VL-3B-GRPO-Thinking
• Original Architecture Paper: Qwen2.5-VL Technical Report

Uses

Direct Use

• Thinking VQA (Visual Question Answering): Solving complex, image-based questions by providing step-by-step explanations.
• Information Extraction & OCR: Reading and structuring text from invoices, charts, scanned documents, and dense text images.
• Agentic Assistant: Serving as a foundation for autonomous agents analyzing mobile or computer UI screens.

Out-of-Scope Use

• Generating misleading content (fake news) or bypassing safety guardrails.
• Deployment in high-risk automated decision-making systems (medical, legal) without human oversight.

Bias, Risks, and Limitations

• Hallucinations: Despite the reasoning fine-tuning, the model may occasionally generate logical thinking steps based on incorrect visual recognition (perceiving the wrong thing in the image).
• Size Limitations: With approximately 3 billion parameters (3B), this model is highly efficient for edge devices and fast inference. However, it may struggle with overly complex, highly specialized reasoning tasks compared to larger 72B models.

Recommendations

Users should be aware of these limitations. A "Human-in-the-loop" approach is strongly recommended when dealing with sensitive data or critical information extraction.

How to Get Started with the Model

Make sure you have the latest version of the transformers library installed. Use the code below to load and run the model:

from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
import torch

# 1. Initialize model and processor
model_id = "Manh1011/Qwen2.5-VL-3B-GRPO-Thinking"
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    model_id, 
    torch_dtype=torch.bfloat16, 
    device_map="auto"
)
processor = AutoProcessor.from_pretrained(model_id)

# 2. Prepare multi-modal inputs
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "[https://example.com/image.jpg](https://example.com/image.jpg)"}, # Replace with actual image URL
            {"type": "text", "text": "Carefully analyze this image and explain step-by-step what is happening."}
        ]
    }
]

# 3. Process the data
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(
    text=[text], 
    images=["[https://example.com/image.jpg](https://example.com/image.jpg)"], 
    return_tensors="pt", 
    padding=True
).to(model.device)

# 4. Generate Output (Inference)
generated_ids = model.generate(**inputs, max_new_tokens=1024)
generated_ids_trimmed = [
    out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]

output_text = processor.batch_decode(
    generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False
)
print(output_text[0])