ViVQA-X_LSTM-Generative

ViVQA-X: LSTM-Generative Baseline Model

This repository contains the LSTM-Generative baseline model from the research paper "An Automated Pipeline for Constructing a Vietnamese VQA-NLE Dataset". The model is designed for Visual Question Answering with Natural Language Explanations (VQA-NLE) in Vietnamese.

Given an image and a question in Vietnamese, it predicts an answer and generates a sentence explaining its reasoning.

Main Project Repository: duongtruongbinh/ViVQA-X

---

🚀 How to Use

This model is a custom PyTorch checkpoint and requires the model class definition from the main project. The intended use is to download the checkpoint and load it using the project's source code.

import torch
from huggingface_hub import hf_hub_download
# You need to have the model definition file, for example: from src.models.baseline_model.vivqax_model import ViVQAX_Model

# Download checkpoint from Hub
checkpoint_path = hf_hub_download(
    repo_id="VLAI-AIVN/ViVQA-X_LSTM-Generative",  # <-- REPLACE WITH YOUR MODEL REPO NAME
    filename="best_model.pth"                 # Checkpoint file name
)

# Load checkpoint
checkpoint = torch.load(checkpoint_path, map_location='cpu')

# Initialize model with structure and vocab from checkpoint
# (This is an example, you need to adjust it to your code)
# model = ViVQAX_Model(
#     vocab_size=len(checkpoint['word2idx']),
#     embed_size=checkpoint['config']['model']['embed_size'],
#     # ... other parameters
# )

# Load trained weights
# model.load_state_dict(checkpoint['model_state_dict'])
# model.eval()

print("Model loaded successfully!")
# Now you can use the model to predict