multimodal-document-retrieval-20260901-model

Multimodal Document Retrieval Baseline Baseline Model

Model Description

This repository contains a small, transparent prototype model for Business documents contain meaning in text, tables, layout, and imagery that text-only retrieval can miss.

The model combines per-label token weights with IDF-weighted evidence retrieval. It was generated for reproducible architecture demonstrations and does not call a hosted LLM.

Evaluation

• Held-out synthetic examples: 4
• Accuracy: 1
• Intended metrics: retrieval_accuracy, modality_coverage, recall_at_3

Intended Use

• Architecture prototyping
• CI and evaluation examples
• Local baseline comparisons
• Educational experimentation

Hugging Face Task Coverage

• visual-document-retrieval
• document-question-answering
• image-to-text
• feature-extraction

Limitations and Risks

The starter dataset contains synthetic textual modality descriptors, not sensitive scanned documents.

The dataset is synthetic and small. Do not use this model for consequential decisions without representative data, expert review, and production-grade evaluation.

Reproducibility

The linked GitHub repository includes train.py, the exact dataset split, evaluation code, and the model JSON format.