Downloads · 30 days
11
5% of all-time downloads
Waffle-v1.0
mehmetkuzucu/Waffle-v1.0
Waffle-v1.0 is a visual question answering model from mehmetkuzucu. Use it for the visual question answering task on the model card, and read the license before you ship it in a product. The card lists the license as cc-by-nc-4.0.
This is a multimodal Visual Question Answering system built for my Bachelor's final project. It combines a Vision Transformer (ViT) image encoder and a SmolLM2 language model using a cross-attention mechanism.
All-time downloads
201
Public
Parameters
223M
891 MB on disk
Likes
4
Public
Repo makeup
Click a slice to open those files.
.safetensors891 MB · 99%
At a glance
- Task
- Visual Question Answering
- License
- cc-by-nc-4.0
- Access
- Public
- Created
- Jun 10, 2025
- Updated
- Jun 10, 2025
- SHA
- d920cb8b
Model card
From the Hugging Face model README
Visual Question Answering (VQA) Model
This is a multimodal Visual Question Answering system built for my Bachelor's final project. It combines a Vision Transformer (ViT) image encoder and a SmolLM2 language model using a cross-attention mechanism.
Model Architecture
- Vision Encoder: Pretrained ViT
- Language Model: SmolLM2-135M
- Fusion: Cross-attention layer aligning vision and language
- Dataset: VQA v2 and LLaVa datasets for training
How to Use
from transformers import AutoProcessor, AutoModelForVisualQuestionAnswering
from PIL import Image
processor = AutoProcessor.from_pretrained("mehmetkuzucu/Waffle-v1.0")
model = AutoModelForVisualQuestionAnswering.from_pretrained("mehmetkuzucu/Waffle-v1.0")
image = Image.open("example.jpg")
question = "What is the person doing?"
inputs = processor(images=image, text=question, return_tensors="pt")
outputs = model(**inputs)
answer = processor.tokenizer.decode(outputs.logits.argmax(-1))
- Task
- Visual Question Answering
- License
- cc-by-nc-4.0
- Created
- Jun 10, 2025
- Updated
- Jun 10, 2025