Downloads · 30 days
4
17% of all-time downloads
bhaskar1707/smolvlm2-bangla-bayanno-chitrojera-lora
smolvlm2-bangla-bayanno-chitrojera-lora is a image-text-to-text model from bhaskar1707. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for peft.
BanglaVLM-v2 is a Bengali Vision-Language Model created by LoRA fine-tuning SmolVLM2-2.2B-Instruct using two Bengali visual question-answering datasets:
Downloads · 30 days
4
17% of all-time downloads
All-time downloads
23
Public
Repo size
116 MB
Likes
0
Public
Click a slice to open those files.
.safetensors108 MB · 82%
From the Hugging Face model README
BanglaVLM-v2 is a Bengali Vision-Language Model created by LoRA fine-tuning SmolVLM2-2.2B-Instruct using two Bengali visual question-answering datasets:
This repository contains the trained LoRA / PEFT adapter weights, evaluation results, predictions, training artifacts, and visualizations from Experiment 2.
| Property | Value |
|---|---|
| Model | BanglaVLM-v2 |
| Base Model | HuggingFaceTB/SmolVLM2-2.2B-Instruct |
| Fine-tuning Method | LoRA / PEFT |
| Training Experiment | Experiment 2 |
| Training Datasets | Bangla-Bayanno + ChitroJera |
| Language | Bengali / Bangla |
| Task | Bengali Visual Question Answering |
| Adapter Type | LoRA Adapter |
| Evaluation Metrics | Exact Match, Normalized Exact Match, ROUGE-L |
BanglaVLM-v2 is the second experiment in a Bengali Vision-Language Model fine-tuning project.
The previous model, BanglaVLM-v1, was trained using the Bangla-Bayanno dataset.
Experiment 2 expands the training setup by incorporating two Bengali VQA datasets:
The objective is to investigate how multi-dataset Bengali VQA training affects performance across different Bengali visual question-answering datasets.
BanglaVLM-v2 is built on:
HuggingFaceTB/SmolVLM2-2.2B-Instruct
The model was fine-tuned using Parameter-Efficient Fine-Tuning through LoRA.
This repository stores the trained adapter weights rather than a fully merged standalone model.
Main adapter files:
adapter_model.safetensorsadapter_config.jsonThe adapter should be loaded together with the original SmolVLM2 base model.
Bangla-Bayanno is one of the Bengali visual question-answering datasets used in Experiment 2.
| Metric | Value |
|---|---|
| Total QA Records | 53,817 |
| Unique Images Referenced | 4,717 |
| Missing Images Detected | 43 |
| Corrupt Images | 0 |
| Empty Questions | 1 |
| Empty Answers | 0 |
| Duplicate QA Pairs | 5,212 |
| Duplicate Image References | 4,703 |
| Invalid Paths | 1 |
ChitroJera is the second Bengali visual question-answering dataset incorporated in Experiment 2.
| Metric | Value |
|---|---|
| Total QA Records | 12,231 |
| Unique Images Referenced | 12,231 |
| Missing Images | 0 |
| Corrupt Images | 0 |
| Empty Questions | 0 |
| Empty Answers | 0 |
| Duplicate QA Pairs | 0 |
| Duplicate Image References | 0 |
| Invalid Paths | 0 |



Raw GPU monitoring data:
logs/gpu_monitor.csv
The models were evaluated on two held-out test sets.
Metrics:
| Model | Exact Match | Normalized Exact Match | ROUGE-L | Test Samples |
|---|---|---|---|---|
| Base SmolVLM2 | 0.00% | 0.00% | 0.23% | 5,004 |
| BanglaVLM-v1 | 33.47% | 33.51% | 34.17% | 5,004 |
| BanglaVLM-v2 | 32.33% | 32.37% | 33.11% | 5,004 |
| Model | Exact Match | Normalized Exact Match | ROUGE-L | Test Samples |
|---|---|---|---|---|
| Base SmolVLM2 | 0.00% | 0.00% | 0.18% | 1,224 |
| BanglaVLM-v1 | 3.10% | 3.10% | 5.34% | 1,224 |
| BanglaVLM-v2 | 6.29% | 6.29% | 9.95% | 1,224 |
The following visualization compares Base SmolVLM2, BanglaVLM-v1, and BanglaVLM-v2.

Experiment 1 trained BanglaVLM-v1 using Bangla-Bayanno.
Experiment 2 trained BanglaVLM-v2 using Bangla-Bayanno and ChitroJera.

The measured effect of multi-dataset training differs across the two evaluation datasets.
Compared with BanglaVLM-v1, BanglaVLM-v2 showed stronger measured performance on the ChitroJera evaluation set while producing slightly lower measured scores on the Bangla-Bayanno evaluation set.
The following visualization contains qualitative Bengali VQA examples generated by BanglaVLM-v2.

| Stage | Time |
|---|---|
| Dataset Download | 4.46 seconds |
| Model Loading | 1005.47 seconds |
| Evaluation | 2.72 hours |
| Training | 0.82 hours |
Raw timing records:
training/timing.jsontraining/timing.csvThis repository contains a PEFT / LoRA adapter.
Load it together with the original SmolVLM2 base model.
from transformers import AutoProcessor, AutoModelForVision2Seq
from peft import PeftModel
base_model_id = "HuggingFaceTB/SmolVLM2-2.2B-Instruct"
adapter_id = "bhaskar1707/smolvlm2-bangla-bayanno-chitrojera-lora"
processor = AutoProcessor.from_pretrained(base_model_id)
base_model = AutoModelForVision2Seq.from_pretrained(base_model_id)
model = PeftModel.from_pretrained(
base_model,
adapter_id
)
If you use this model or its associated results, please cite:
Bhaskar Pal.
BanglaVLM-v2: LoRA Fine-Tuning SmolVLM2 for Bengali Visual Question Answering
using Bangla-Bayanno and ChitroJera.
Hugging Face Model Repository.
Please refer to the licenses and usage terms of the SmolVLM2 base model, Bangla-Bayanno dataset, and ChitroJera dataset before using this model or its associated artifacts.