qwen2-7b-instruct-trl-sft-ChartQA

Qwen2-VL-7B-Instruct - ChartQA SFT (LoRA)

A PEFT LoRA adapter that fine-tunes Qwen/Qwen2-VL-7B-Instruct for chart understanding and visual question answering on the ChartQA dataset, trained with the TRL SFTTrainer.

Given a chart image and a natural-language question, the model reasons over the chart's visual elements (bars, lines, axes, labels) to produce an answer.

Training

Usage

from transformers import Qwen2VLForConditionalGeneration, AutoProcessor
from peft import PeftModel
import torch

base = Qwen2VLForConditionalGeneration.from_pretrained(
    "Qwen/Qwen2-VL-7B-Instruct", torch_dtype=torch.bfloat16, device_map="auto"
)
model = PeftModel.from_pretrained(base, "lognat0704/qwen2-7b-instruct-trl-sft-ChartQA")
processor = AutoProcessor.from_pretrained("Qwen/Qwen2-VL-7B-Instruct")

messages = [{
    "role": "user",
    "content": [
        {"type": "image", "image": "chart.png"},
        {"type": "text", "text": "What is the highest value in the chart?"},
    ],
}]
# See the Qwen2-VL docs for full multimodal preprocessing.

Notes

• This is an adapter, not a full model — it must be loaded on top of Qwen/Qwen2-VL-7B-Instruct.
• Intended for chart/plot QA; not evaluated for general-purpose VQA.