Downloads · 30 days
13
3% of all-time downloads
alexshengzhili/llava-v1.5-13b-dpo
llava-v1.5-13b-dpo is a text generation model from alexshengzhili. Use it when you need the model to write or continue text. It is set up for transformers.
[Arxiv paper] [GitHub] [Data] [Model] [Data]
Downloads · 30 days
13
3% of all-time downloads
All-time downloads
372
Public
Repo size
53.4 GB
Likes
5
Public
Click a slice to open those files.
.bin26.7 GB · 100%
From the Hugging Face model README
[Arxiv paper] [GitHub] [Data] [Model] [Data]
Developers: Shengzhi Li (TIFIN), Rongyu Lin (KAUST), Shichao Pei (University of Massachusetts Boston)
Affiliations: TIFIN, KAUST, University of Massachusetts Boston
Contact Information: [email protected], [email protected], [email protected]
The llava-v1.5-13b-dpo model is designed to enhance the instruction-following capabilities of multi-modal large language models (MLLMs), particularly in scenarios where visual instruction tuning might degrade language proficiency. This model leverages a novel Direct Preference Optimization (DPO) method, along with a curated 6K-entry VQA preference dataset, to achieve superior performance on multi-modal tasks and benchmarks.
The MM-LLM-DPO model was trained using a lightweight (6k entries) VQA preference dataset, where answers were annotated for 5 quality metrics in a granular fashion. The dataset was designed to address the diversity and complexity gap typically observed in VQA datasets.
The model demonstrates significant improvements over baseline models like Vicuna and LLaVA on various benchmarks:
| Model Name | MM-Vet | LLaVA-bench | PoPe | MM-Bench | MT-bench | AlpacaEval |
|---|---|---|---|---|---|---|
| Vicuna-1.5-13b [16] | - | - | - | - | 6.57 | 81.4 |
| LLaVA-1.5-13b [10] | 36.3 | 73.1 | 0.859 | 67.4 | 5.99 | 79.3 |
| LLaVA-RLHF-13b [23] | 37.2 | 76.8 | 0.869 | 60.1 | 6.18 | 81.0 |
| Standard SFT | 36.5 | 63.7 | 0.850 | 65.4 | 5.01 | 50.2 |
| SteerLM | 35.2 | 67.0 | 0.878 | 65.1 | 5.70 | 68.8 |
| Rejection-sampling | 38.0 | 70.6 | 0.883 | 67.6 | 6.22 | 74.9 |
| llava-v1.5-13b-dpo | 41.2 | 79.1 | 0.870 | 66.8 | 6.73 | 86.4 |
*We applied the last four Standard sft, SteerLM, Rejection Sampling and DPO, and found DPO to be most performant
This model was developed with a focus on mitigating modality conflict and catastrophic forgetting in MLLMs. Users are encouraged to consider the potential biases and limitations inherent in the training data and model outputs, especially when deploying the model in diverse and sensitive contexts.
This work was made possible through the contributions of Shengzhi Li, Rongyu Lin, and Shichao Pei, and supported by their respective institutions.
Please cite this work as:
@misc{li2024multimodal,
title={Multi-modal preference alignment remedies regression of visual instruction tuning on language model},
author={Shengzhi Li and Rongyu Lin and Shichao Pei},
year={2024},
eprint={2402.10884},
archivePrefix={arXiv},
primaryClass={cs.CL}
}