qwen3.5-0.8b-tablevqa-lora

qwen3.5-0.8b-tablevqa-lora

A LoRA adapter for Qwen/Qwen3.5-0.8B, from the MiniVLMDocEval Part-2 study. It targets the model's measured weakness on visual table lookup (Wikipedia-style VWTQ) via bf16 LoRA with the vision encoder frozen.

⚠️ This is a research artifact, not a recommended checkpoint

This adapter failed the project's pre-registered no-regression gate: instead of a targeted gain it catastrophically regressed the base model (TableVQABench 50.1 → 19.6, DocVQA 89.3 → 50.8, InfoVQA 62.3 → 27.8). The diagnosis is a train/evaluation answer-style mismatch — the synthetic training source supplies verbose, full-sentence answers while the benchmarks are scored on short cell values/spans — which collapsed the model's output distribution. It is published for transparency and reproducibility of the negative result; do not use it as an improved model. The full diagnosis and a corrected recipe are in the technical report at https://github.com/SajjadPSavoji/MiniVLMDocEval.

Usage

from transformers import AutoModelForImageTextToText, AutoProcessor
from peft import PeftModel

base = AutoModelForImageTextToText.from_pretrained("Qwen/Qwen3.5-0.8B", torch_dtype="bfloat16")
model = PeftModel.from_pretrained(base, "savoji/qwen3.5-0.8b-tablevqa-lora").eval()
processor = AutoProcessor.from_pretrained("savoji/qwen3.5-0.8b-tablevqa-lora")

Training

bf16 LoRA, vision encoder frozen, adapters on the language + gated-delta projections; r=16, alpha=32, lr=0.0001, epochs=1.0, batch=1xgrad_accum=16. One epoch on a ~7k VWTQ-weighted mixture with ~25% general-document replay. Reproduce with scripts/train_lora_qwen.py.