Qwen3-VL-TimeTravel

Qwen3-VL-TimeTravel Fine-tuned Model

This model is a fine-tuned version of the Qwen3-VL-8B-Instruct model using the Unsloth library on the MBZUAI/TimeTravel dataset.

Model Details

• Base Model: Qwen3-VL-8B-Instruct-unsloth-bnb-4bit
• Fine-tuning Library: Unsloth
• Dataset: MBZUAI/TimeTravel
• Task: Image description generation for historical artifacts.

Training

The model was trained for 4 epochs with a batch size of 8 (4 per device with 8 gradient accumulation steps). LoRA adapters were used for parameter efficient finetuning, targeting both vision and language layers, as well as attention and MLP modules.

Dataset

<div align="center" style="margin-top:10px;"> <img src='asset/logo.png' align="left" width="7%" /> </div> <div style="margin-top:50px;"> <h3 style="font-size: 20px; margin: 0;"> TimeTravel: A Comprehensive Benchmark to Evaluate LMMs on Historical and Cultural Artifacts</h3> </div> <div align="center" style="margin-top:10px;">

Sara Ghaboura ^*   Ketan More ^*   Retish Thawkar   Wafa Alghallabi   Omkar Thawakar   <br> Fahad Shahbaz Khan   Hisham Cholakkal   Salman Khan   Rao M. Anwer<br> <em> ^{*Equal Contribution} </em> <br>

</div> <div align="center" style="margin-top:10px;"> [](https://arxiv.org/abs/2502.14865) [](https://mbzuai-oryx.github.io/TimeTravel/)

🏛 TimeTravel Taxonomy and Diversity

<p align="left"> TimeTravel Taxonomy maps artifacts from 10 civilizations, 266 cultures, and 10k+ verified samples for AI-driven historical analysis. </p> <p align="center"> <img src="asset/Intro.png" width="750px" height="400px" alt="tax" style="margin-right: 2px";/> </p> </div> <br>

🌟 Key Features

TimeTravel is the first large-scale, open-source benchmark designed to evaluate Large Multimodal Models (LMMs) on historical and cultural artifacts. It covers:

• 266 Cultural Groups across 10 Historical Regions
• 10,000+ Expert-Verified Artifact Samples
• Multimodal Image-Text Dataset for AI-driven historical research
• A publicly available dataset and evaluation framework to advance AI applications in history and archaeology.

<br>

🔄 TimeTravel Creation Pipeline

The TimeTravel dataset follows a structured pipeline to ensure the accuracy, completeness, and contextual richness of historical artifacts.<br>

<p align="center"> <img src="asset/pipe_last.png" width="750px" height="150px" alt="pipeline" style="margin-right: 2px";/> </p>

Our approach consists of four key phases:

• Data Selection: Curated 10,250 artifacts from museum collections, spanning 266 cultural groups, with expert validation to ensure historical accuracy and diversity.<br>
• Data Cleaning: Addressed missing or incomplete metadata (titles, dates, iconography) by cross-referencing museum archives and academic sources, ensuring data consistency.<br>
• Generation & Verification: Used GPT-4o to generate context-aware descriptions, which were refined and validated by historians and archaeologists for authenticity.<br>
• Data Aggregation: Standardized and structured dataset into image-text pairs, making it a valuable resource for AI-driven historical analysis and cultural heritage research.<br>

<br>

🏆 TimeTravel Evaluation

The table below showcases the performance comparison of various closed and open-source models on our proposed TimeTravel benchmark.

<div align="center";> <h5> <table> <thead> <tr style="background-color: #EBD9B3; color: white;"> <th>Model</th> <th>BLEU</th> <th>METEOR</th> <th>ROUGE-L</th> <th>SPICE</th> <th>BERTScore</th> <th>LLM-Judge</th> </tr> </thead> <tbody> <tr> <td>GPT-4o-0806</td> <td><b>0.1758🏅</b></td> <td>0.2439</td> <td><b>0.1230🏅</b></td> <td><b>0.1035🏅</b></td> <td><b>0.8349🏅</b></td> <td><b>0.3013🏅</b></td> </tr> <tr> <td>Gemini-2.0-Flash</td> <td>0.1072</td> <td>0.2456</td> <td>0.0884</td> <td>0.0919</td> <td>0.8127</td> <td>0.2630</td> </tr> <tr> <td>Gemini-1.5-Pro</td> <td>0.1067</td> <td>0.2406</td> <td>0.0848</td> <td>0.0901</td> <td>0.8172</td> <td>0.2276</td> </tr> <tr> <td>GPT-4o-mini-0718</td> <td>0.1369</td> <td><b>0.2658🏅</b></td> <td>0.1027</td> <td>0.1001</td> <td>0.8283</td> <td>0.2492</td> </tr> <tr> <td>Llama-3.2-Vision-Inst</td> <td>0.1161</td> <td>0.2072</td> <td>0.1027</td> <td>0.0648</td> <td>0.8111</td> <td>0.1255</td> </tr> <tr> <td>Qwen-2.5-VL</td> <td>0.1155</td> <td>0.2648</td> <td>0.0887</td> <td>0.1002</td> <td>0.8198</td> <td>0.1792</td> </tr> <tr> <td>Llava-Next</td> <td>0.1118</td> <td>0.2340</td> <td>0.0961</td> <td>0.0799</td> <td>0.8246</td> <td>0.1161</td> </tr> </tbody> </table> </h5> <p> <div align="center";> <h5> <table> <thead> <tr style="background-color: #EBD9B3; color: white;"> <th>Model</th> <th>India</th> <th>Roman Emp.</th> <th>China</th> <th>British Isles</th> <th>Iran</th> <th>Iraq</th> <th>Japan</th> <th>Cent. America</th> <th>Greece</th> <th>Egypt</th> </tr> </thead> <tbody> <tr> <td>GPT-4o-0806</td> <td><b>0.2491🏅</b></td> <td><b>0.4463🏅</b></td> <td><b>0.2491🏅</b></td> <td><b>0.1899🏅</b></td> <td><b>0.3522🏅</b></td> <td><b>0.3545🏅</b></td> <td><b>0.2228🏅</b></td> <td><b>0.3144🏅</b></td> <td><b>0.2757🏅</b></td> <td><b>0.3649🏅</b></td> </tr> <tr> <td>Gemini-2.0-Flash</td> <td>0.1859</td> <td>0.3358</td> <td>0.2059</td> <td>0.1556</td> <td>0.3376</td> <td>0.3071</td> <td>0.2000</td> <td>0.2677</td> <td>0.2582</td> <td>0.3602</td> </tr> <tr> <td>Gemini-1.5-Pro</td> <td>0.1118</td> <td>0.2632</td> <td>0.2139</td> <td>0.1545</td> <td>0.3320</td> <td>0.2587</td> <td>0.1871</td> <td>0.2708</td> <td>0.2088</td> <td>0.2908</td> </tr> <tr> <td>GPT-4o-mini-0718</td> <td>0.2311</td> <td>0.3612</td> <td>0.2207</td> <td>0.1866</td> <td>0.2991</td> <td>0.2632</td> <td>0.2087</td> <td>0.3195</td> <td>0.2101</td> <td>0.2501</td> </tr> <tr> <td>Llama-3.2-Vision-Inst</td> <td>0.0744</td> <td>0.1450</td> <td>0.1227</td> <td>0.0777</td> <td>0.2000</td> <td>0.1155</td> <td>0.1075</td> <td>0.1553</td> <td>0.1351</td> <td>0.1201</td> </tr> <tr> <td>Qwen-2.5-VL</td> <td>0.0888</td> <td>0.1578</td> <td>0.1192</td> <td>0.1713</td> <td>0.2515</td> <td>0.1576</td> <td>0.1771</td> <td>0.1442</td> <td>0.1442</td> <td>0.2660</td> </tr> <tr> <td>Llava-Next</td> <td>0.0788</td> <td>0.0961</td> <td>0.1455</td> <td>0.1091</td> <td>0.1464</td> <td>0.1194</td> <td>0.1353</td> <td>0.1917</td> <td>0.1111</td> <td>0.0709</td> </tr> </tbody> </table> </h5> <p> <div align="left"></div> <br>

🖼 TimeTravel Examples

<p align="left"> The figure illustrates the cultural and material diversity of the TimeTravel dataset. </p> <p align="center"> <img src="asset/fig0.png" width="1000px" height="250px" alt="tax" style="margin-right: 2px";/> </p> <div align="left";> <br> <div class="tree-container"> <h2>📂 TimeTravle Dataset Schema</h2> <div class="tree"> <ul> <li><span class="leaf">📷 Image</span> (image)</li> <li><span class="leaf">🔹 id</span> (string)</li> <li><span class="leaf">📅 Production date</span> (string)</li> <li><span class="leaf">📍 Find spot</span> (string)</li> <li><span class="leaf">🔸 Materials</span> (string)</li> <li><span class="leaf">🛠 Technique</span> (string)</li> <li><span class="leaf">📝 Inscription</span> (string)</li> <li><span class="leaf">🎭 Subjects</span> (string)</li> <li><span class="leaf">📛 Assoc name</span> (string)</li> <li><span class="leaf">🏛 Culture</span> (string)</li> <li><span class="leaf">📂 Section</span> (string)</li> <li><span class="leaf">🌍 Place</span> (string)</li> <li><span class="leaf">📝 description</span> (string)</li> </ul> </div> </div> </div> <br>

📚 Citation

<p align="left"> If you use TimeTravle dataset in your research, please consider citing: </p> <div align="left">

@misc{ghaboura2025timetravelcomprehensivebenchmark,
      title={Time Travel: A Comprehensive Benchmark to Evaluate LMMs on Historical and Cultural Artifacts}, 
      author={Sara Ghaboura and Ketan More and Ritesh Thawkar and Wafa Alghallabi and Omkar Thawakar and Fahad Shahbaz Khan and Hisham Cholakkal and Salman Khan and Rao Muhammad Anwer},
      year={2025},
      eprint={2502.14865},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2502.14865}, 
}

</div> </div>

Usage

This model can be used for generating descriptions of historical artifact images.

<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>