Qwen2-VL-2B-DPO-LoRA

Qwen2-VL-2B-Instruct DPO (Hallucination Mitigated)

This model is a Direct Preference Optimization (DPO) aligned version of Qwen/Qwen2-VL-2B-Instruct, fine-tuned using LoRA on the HA-DPO preference dataset to significantly reduce visual hallucinations.

Quickstart & Usage

import torch
from transformers import Qwen2VLForConditionalGeneration, AutoProcessor
from qwen_vl_utils import process_vision_info

model_id = "srianna/Qwen2-VL-2B-DPO-LoRA"

model = Qwen2VLForConditionalGeneration.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
processor = AutoProcessor.from_pretrained(model_id)

messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "image": "path/to/your/image.jpg"},
            {"type": "text", "text": "Describe this image in detail."},
        ],
    }
]

text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
image_inputs, video_inputs = process_vision_info(messages)

inputs = processor(
    text=[text],
    images=image_inputs,
    videos=video_inputs,
    padding=True,
    return_tensors="pt"
).to(model.device)

with torch.no_grad():
    output_ids = model.generate(
        **inputs,
        max_new_tokens=512,
        do_sample=False,
        repetition_penalty=1.2,
    )

generated_ids = output_ids[:, inputs.input_ids.shape[1]:]
print(processor.batch_decode(generated_ids, skip_special_tokens=True)[0])

Training Details

• Base Model: Qwen/Qwen2-VL-2B-Instruct
• Method: DPO (Direct Preference Optimization) + LoRA (BF16)
• Dataset: HA-DPO (juliozhao/hadpo-data)