unsloth-pret-lora

✨ Example Use

from unsloth import FastVisionModel
from transformers import AutoTokenizer

model, tokenizer = FastVisionModel.from_pretrained(
    "unsloth/llama-3-vision-8b",
    adapter="alinasdkey/pret_lora",  # <--- Your LoRA adapter!
    load_in_4bit=True,
)

FastVisionModel.for_inference(model)

image = Image.open("your_graph.png")
instruction = "Describe this graph in detail."

messages = [{"role": "user", "content": [{"type": "image"}, {"type": "text", "text": instruction}]}]
input_text = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
inputs = tokenizer(input_text, return_tensors="pt").to("cuda")

outputs = model.generate(**inputs, max_new_tokens=100)
caption = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(caption)
base_model: unsloth/llama-3.2-11b-vision-instruct-unsloth-bnb-4bit
library_name: peft
license: cc-by-4.0
datasets:
- alinasdkey/graphtrain91
language:
- en
--

### Training
LoRA Adapter trained using Unsloth, Free Colab 

Dataset: 91 image-caption pairs focused on diverse graph types

Format: JSONL with keys: "image", "input", "text"

#Use Cases
Educational tools for the visually impaired

Graph captioning for automated note-taking

AI tutoring in STEM subjects

### 📊 Evaluation

This LoRA adapter has not been benchmarked with formal quantitative metrics yet, but has been **qualitatively assessed** for:
- Relevance and accuracy of graph descriptions
- Clarity and detail of language
- Improvement over the base model's zero-shot captions

More structured evaluations (e.g., BLEU, CIDEr) may be added in future updates.

- PEFT 0.15.2