qwen25vl-7b-perdataset-derivatives

Qwen2.5-VL-7B — Per-Dataset Fine-Tuning Derivatives (LoRA + Full FT)

10 fine-tuned derivatives of Qwen/Qwen2.5-VL-7B-Instruct, one LoRA and one full fine-tune for each of 5 vision-language datasets. Produced as clean single-dataset transfer targets for a visual-fingerprint transfer study.

⚠️ Research artifacts. Each model is trained on a 3,000-example slice of one dataset for 3 epochs — these are experimental derivatives, not production models.

Shared training recipe (all 10)

• Base: Qwen/Qwen2.5-VL-7B-Instruct
• Frozen: vision tower + multimodal projector (freeze_vision_tower: true, freeze_multi_modal_projector: true) — only the language-model backbone is trained.
• Data: max_samples = 3000 per dataset, 3 epochs, cutoff_len = 2048.
• Framework: LLaMA-Factory 0.9.3, transformers 4.52.4.
• LoRA variant: target q_proj, v_proj; rank 16, alpha 32; lr 2e-4; trainable ≈ 0.06 % of params.
• Full variant: DeepSpeed ZeRO-3; lr 1e-5; image_max_pixels = 401408; trainable = 100 % of the LM (vision/projector still frozen).

Contents

Datasets: MathV360K, PaintingForm, ST-VQA, TextVQA, Visual7W (single-dataset VQA-style instruction tuning).

Usage

LoRA (needs the base model + PEFT):

from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
from peft import PeftModel
base = Qwen2_5_VLForConditionalGeneration.from_pretrained("Qwen/Qwen2.5-VL-7B-Instruct", torch_dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(base, "DSA1/qwen25vl-7b-perdataset-derivatives", subfolder="lora/textvqa")
proc = AutoProcessor.from_pretrained("Qwen/Qwen2.5-VL-7B-Instruct")

Full FT (self-contained weights):

from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
model = Qwen2_5_VLForConditionalGeneration.from_pretrained("DSA1/qwen25vl-7b-perdataset-derivatives", subfolder="full/textvqa", torch_dtype="auto", device_map="auto")
proc = AutoProcessor.from_pretrained("Qwen/Qwen2.5-VL-7B-Instruct")   # processor/chat-template from base

Notes

• train_loss is the mean training loss over all optimizer steps (the LoRA runs ~300 steps for the 4 smaller datasets and ~1014 for textvqa; the full runs are 129 steps at global batch 64).
• full/textvqa was taken from the run's checkpoint-129 (the final training step): its top-level save was interrupted by a cluster preemption, but checkpoint-129 holds the complete 4-shard, 16.6 GB, 729-tensor model. It is the fully-trained end-of-run model.
• Full-FT models ship the LM weights only; load the processor / chat template from the base Qwen/Qwen2.5-VL-7B-Instruct.

License

Inherits the base model's license (Qwen/Qwen2.5-VL-7B-Instruct). Datasets retain their respective licenses.