qwen2-vl-8b-lora-finetune

Usage

This repository contains LoRA adapters only.

Base model:

• Qwen2-VL-8B-Instruct

## load the model from my own huggingface 
from unsloth import FastVisionModel

model, tokenizer = FastVisionModel.from_pretrained(
    "Rohit-Katkar2003/qwen2-vl-8b-lora-finetune",
    load_in_4bit=True
)

FastVisionModel.for_inference(model) # Enable for inference!
question = "what are the components in diagram give me response in markdown format with all the connection and components?"
url = "https://instrumentationtools.com/wp-content/uploads/2016/09/instrumentationtools.com_piping-and-instrumentation-diagram.jpg"
res = requests.get(url)
image = Image.open(BytesIO(res.content)) 
messages = [
    {"role": "user", "content": [
        {"type": "image"},
        {"type": "text", "text": question}
    ]}
]
input_text = tokenizer.apply_chat_template(messages, add_generation_prompt = True)
inputs = tokenizer(
    image,
    input_text,
    add_special_tokens = False,
    return_tensors = "pt",
).to("cuda")

from transformers import TextStreamer
text_streamer = TextStreamer(tokenizer, skip_prompt = True)
_ = model.generate(**inputs, streamer = text_streamer, max_new_tokens = 1024,
                   use_cache = True, temperature = 0.3, min_p = 0.1)
                                                                            

Uploaded model

• Developed by: Rohit-Katkar2003
• License: apache-2.0
• Finetuned from model : unsloth/Qwen3-VL-8B-Instruct-unsloth-bnb-4bit

This qwen3_vl model was trained 2x faster with Unsloth

<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>