DiaNA-Plus

Towards Interactive Chat-based Person Retrieval with Dialogue-aware Evolution

Chat-based person retrieval (ChatPR) aims to identify a target person through progressively evolving multi-turn dialogues. Building on our CVPR 2025 work, Chat-based Person Retrieval via Dialogue-Refined Cross-Modal Alignment (📄 Paper · 💻 Code), DiaNA+ further improves cross-modal alignment under sparse conversational cues and evolving dialogue states.

The implementation of DiaNA+ is available at: 👉 https://github.com/Flame-Chasers/DiaNA-Plus

<p align="center"> <img src="https://huggingface.co/byougert/DiaNA-Plus/resolve/main/assets/architecture.png" alt="DiaNA+ architecture" width="95%"> </p>

📖 Overview

DiaNA+ aligns evolving dialogue queries with person images through:

• Adaptive Attribute Refinement (AAR) for aggregating sparse person-related cues across dialogue turns.
• Dialogue-aware Visual Calibration (DVC) for adapting visual representations to the current dialogue state.
• Random Round Retaining (R3) for improving robustness across different interaction stages.

The model uses Swin Transformer V2-B as the image encoder and Llama 3.2-1B as the dialogue encoder.

🤗 Model Zoo

🚀 Usage

Clone the implementation and download the model repository:

git clone https://github.com/Flame-Chasers/DiaNA-Plus.git
hf download byougert/DiaNA-Plus --local-dir checkpoints/DiaNA-Plus

Load a finetuned checkpoint:

from diana import load_diana

model, tokenizer = load_diana(
    pretrained="checkpoints/DiaNA-Plus/finetune/ChatPedes-CUHK",
    device="cuda",
)
model.eval()

To use another checkpoint, replace ChatPedes-CUHK with ChatPedes-ICFG or ChatPedes-RSTP. The pretrained checkpoint is located at checkpoints/DiaNA-Plus/pretrain.

✨ Citation

@InProceedings{bai2025chat,
    author    = {Bai, Yang and Ji, Yucheng and Cao, Min and Wang, Jinqiao and Ye, Mang},
    title     = {Chat-based Person Retrieval via Dialogue-Refined Cross-Modal Alignment},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
    pages     = {3952--3962},
    month     = {June},
    year      = {2025}
}

📄 License

This model is released under the MIT License.