Downloads · 30 days
3
14% of all-time downloads
DrMajid/qwen2-vl-2b-ai2d-qlora
qwen2-vl-2b-ai2d-qlora is a image-text-to-text model from DrMajid. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as apache-2.0.
This repository contains a PEFT/LoRA adapter for Qwen/Qwen2-VL-2B-Instruct, fine-tuned on the AI2D subset of HuggingFaceM4/thecauldron.
Downloads · 30 days
3
14% of all-time downloads
All-time downloads
22
Public
Repo size
20.2 MB
Likes
0
Public
Click a slice to open those files.
.json11.4 MB · 57%
From the Hugging Face model README
This repository contains a PEFT/LoRA adapter for
Qwen/Qwen2-VL-2B-Instruct, fine-tuned on the AI2D subset of
HuggingFaceM4/the_cauldron.
Evaluation used a fixed seed-42 80/20 split. The adapter was trained only on the 1,947 training records and evaluated against the unchanged base model on 487 held-out records.
| Model | Exact-choice accuracy | Valid answers |
|---|---|---|
| Qwen2-VL-2B-Instruct | 93.02% | 487/487 |
| Qwen2-VL-2B-Instruct + adapter | 95.07% | 487/487 |
The absolute improvement was +2.05 percentage points. A paired bootstrap with 5,000 resamples gave a 95% confidence interval of +0.62 to +3.70 percentage points.
Answers were generated greedily with at most 16 new tokens and parsed as an exact multiple-choice response from A through D.
| Setting | Value |
|---|---|
| Method | 4-bit NF4 QLoRA |
| LoRA rank / alpha | 16 / 32 |
| LoRA dropout | 0.05 |
| Target modules | q_proj, k_proj, v_proj, o_proj |
| Epochs | 3 |
| Learning rate | 1e-4 |
| Effective batch size | 8 |
| Maximum sequence length | 512 |
| Training records | 1,947 |
| Training time | 39 minutes 52 seconds |
| GPU | NVIDIA GeForce RTX 3070 8 GB |
| PyTorch | 2.13.0+cu130 |
| Transformers | 5.14.1 |
Loss was applied only to assistant-completion tokens. The vision input was bounded between 16 and 256 image patches.
from peft import PeftModel
from transformers import AutoModelForImageTextToText, AutoProcessor
base_id = "Qwen/Qwen2-VL-2B-Instruct"
adapter_id = "YOUR_ACCOUNT/YOUR_REPOSITORY"
processor = AutoProcessor.from_pretrained(base_id)
model = AutoModelForImageTextToText.from_pretrained(base_id, device_map="auto")
model = PeftModel.from_pretrained(model, adapter_id)
This adapter was optimized and evaluated on AI2D multiple-choice diagram questions. The result supports improvement on this fixed held-out split only; it does not establish general VLM improvement. It was not trained using the project's production 7B/FSDP or DPO configurations. AI2D and each component of The Cauldron remain subject to their source licensing conditions.