MMDuet2

<div align="center"> <h1 style="font-size: 32px; font-weight: bold;"> MMDuet2: Enhancing Proactive Interaction of Video MLLMs with Multi-Turn Reinforcement Learning </h1> <br> <p align="left"> 📖 <a href="https://arxiv.org/abs/xxx" target="_blank">Paper</a> · ⭐ <a href="https://github.com/yellow-binary-tree/MMDuet2" target="_blank">GitHub</a> · 📊 <a href="https://huggingface.co/datasets/wangyueqian/MMDuet2-data" target="_blank">Dataset</a> · 🤗 <a href="https://huggingface.co/wangyueqian/MMDuet2" target="_blank">Checkpoints</a> </p> </div>

Key Features:

• MMDuet2 is a Video MLLM for proactive interaction, which means that it can not only reply right after the user's turn, but also at any approprite and timely moment during the video playback.

• With only a 3B model, MMDuet2 is lightweight and fast for real-time interaction.

• Responses are neither too sparse nor too dense and repetitive, which was a common issue in previous works.

(Demo Video Here)

Usage

MMDuet2 is post-trained from Qwen2.5-VL-3B-Intruct with enhanced proactive interaction ability.

Though this model can be used in the exact same way as Qwen2.5-VL, we recommend using the demo and evaluation code provided in MMDuet2 Github Repo to experience its ability of actively generating replies while a video is playing online.