Downloads · 30 days
8
25% of all-time downloads
multimodal-rag-system
Hamza66628/multimodal-rag-system
multimodal-rag-system is a machine learning model from Hamza66628. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for sentence-transformers. The card lists the license as mit.
This repository contains a complete Multimodal Retrieval-Augmented Generation (RAG) system that combines text and image search with LLM-based answer generation.
All-time downloads
32
Public
Repo size
1.1 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.faiss831 KB · 74%
At a glance
- Library
- sentence-transformers
- License
- mit
- Access
- Public
- Created
- Dec 12, 2025
- Updated
- Dec 12, 2025
- SHA
- 52d563ea
Model card
From the Hugging Face model README
Multimodal RAG System
This repository contains a complete Multimodal Retrieval-Augmented Generation (RAG) system that combines text and image search with LLM-based answer generation.
System Components
- Text Embeddings: Sentence-BERT (all-MiniLM-L6-v2) - 384 dimensions
- Image Embeddings: CLIP (ViT-B/32) - 512 dimensions
- Vector Database: FAISS indices for efficient similarity search
- LLM: Mistral-7B-Instruct (4-bit quantized)
- Total Vectors: 446 (161 text + 285 images)
Files
text_index.faiss: FAISS index for text embeddingsimage_index.faiss: FAISS index for image embeddingstext_metadata.pkl: Metadata for text chunks (source, page, content)image_metadata.pkl: Metadata for images (source, page, image_id)config.json: System configurationimage_summary.json: Reference summary of images
Usage
See the load cells in the notebook for loading and using this RAG system.
Features
- Semantic text search
- Cross-modal image search (text query → image results)
- Multiple prompting strategies (Standard, Chain-of-Thought, Few-shot, Zero-shot)
- Source attribution and traceability
- Real-time answer generation
- Library
- sentence-transformers
- License
- mit
- Created
- Dec 12, 2025
- Updated
- Dec 12, 2025