smolvlm2-bangla-bayanno-lora

SmolVLM2-2.2B-Instruct — LoRA fine-tuned on Bangla-Bayanno (BanglaVLM-v1)

LoRA adapter for HuggingFaceTB/SmolVLM2-2.2B-Instruct, fine-tuned for Bengali Visual Question Answering on the Bangla-Bayanno dataset.

Trained on a single Colab T4 GPU (16GB) with a constrained time/step budget (LoRA r=16, False QLoRA, fp16, image-splitting disabled for speed).

Measured results (full held-out test set, n=5004)

Per-answer-type breakdown (BanglaVLM-v1):

Caveats: Trained on a ~7,000-example subset (not the full 42k+ train split) for ~1,000 optimizer steps to fit Colab's free-tier time/session limits. Descriptive-answer accuracy remains low; do not extrapolate beyond these measured numbers.

Usage

from transformers import AutoProcessor, AutoModelForImageTextToText
from peft import PeftModel
import torch

base = AutoModelForImageTextToText.from_pretrained(
    "HuggingFaceTB/SmolVLM2-2.2B-Instruct", torch_dtype=torch.float16
)
model = PeftModel.from_pretrained(base, "bhaskar1707/smolvlm2-bangla-bayanno-lora")
processor = AutoProcessor.from_pretrained("bhaskar1707/smolvlm2-bangla-bayanno-lora", do_image_splitting=False)

Result charts

Dataset analysis (answer-type distribution, question/answer length, split sizes)

GPU utilization / VRAM usage during training

Baseline vs fine-tuned comparison, accuracy by answer type

Qualitative examples