qwen35-4b-docvqa-lora

Qwen3.5-4B DocVQA LoRA (v2_r32_mlp)

LoRA adapters fine-tuning Qwen/Qwen3.5-4B for document visual question answering. Trained with Unsloth on 2,000 curated DocVQA samples from the train split.

Usage (critical: disable thinking)

from unsloth import FastVisionModel
from peft import PeftModel

base, processor = FastVisionModel.from_pretrained("Qwen/Qwen3.5-4B", load_in_4bit=True)
FastVisionModel.for_inference(base)
model = PeftModel.from_pretrained(base, "hxcsa/qwen35-4b-docvqa-lora")
FastVisionModel.for_inference(model)

# Qwen3.5 is a thinking-style model — disable thinking for concise answers
text = processor.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=False,
    enable_thinking=False,  # ← essential
)
inputs = processor(images=image, text=text, add_special_tokens=False, return_tensors="pt").to("cuda")
out = model.generate(**inputs, max_new_tokens=64, do_sample=False)
answer = processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True).strip()

Results (ANLS on 300 DocVQA validation, seed 1234, 1344px)

+0.26 ANLS over zero-shot. Clean eval (trained on train, tested on val).

Training

• Base: Qwen/Qwen3.5-4B (4-bit NF4)
• LoRA: r=32, α=32, dropout 0 — attention (q/k/v/o) + MLP (gate/up/down)
• ViT frozen; loss on assistant tokens only
• 2 epochs, 2,000 samples, eff. batch 8, LR 2e-4 cosine, bf16
• 5% holdout eval → best checkpoint auto-restored
• ~34 min on A6000 48 GB, peak VRAM 4.8 GB

Data

Trained on hxcsa/docvqa-2k-train (2,000 samples from DocVQA train split, quality-filtered, 1344px max edge). The validation ANLS uses the clean DocVQA validation split.

Files

• adapter_model.safetensors — LoRA weights (34 MB)
• tokenizer.json / tokenizer_config.json / chat_template.jinja — Qwen3.5 template
• processor_config.json — vision processor config