LISA_Plus_7b

LISA++: An Improved Baseline for Reasoning Segmentation with Large Language Model

🤗Data | 📄Paper | 🚀Code | 💻Model | 🔥Citation

This model could be used for instance segmentation

Example

<table class="center"> <tr> <td style="text-align:center;"><b>Input</b></td> <td style="text-align:center;"><b>Output</b></td> </tr> <tr> <td><img src="https://raw.githubusercontent.com/dvlab-research/LISA/lisa_plus/imgs/camera_lens.jpg"></td> <td><img src="https://raw.githubusercontent.com/dvlab-research/LISA/lisa_plus/vis_output/camera_lens_masked_img.jpg"></td> </tr> <tr> <td width="50%" style="text-align:center;color:gray;"> Please describe the two cameras. <span style="font-weight: bold; color: #4b8b3b;">+ DEFAULT_INST_CONV_SEG</span> </td> <td width=50% style="text-align:center;">" The image depicts two vibrant camera models resting on a surface. The larger of the two[ [SEG] ] is situated towards the front of the image, characterized by its prominent lens and sturdy build, indicating it could be a D- sl or E- within the camera brand lineup. The other[ [SEG] ], smaller in size, also a camera, is located further back in the image and is partially obscured by the larger one, characterized by its smaller lens and less prominent features. Both are placed on a surface suggesting they might be on display or in use."</td> </tr> <tr> <td><img src="https://raw.githubusercontent.com/dvlab-research/LISA/lisa_plus/imgs/blackpink.jpg"></td> <td><img src="https://raw.githubusercontent.com/dvlab-research/LISA/lisa_plus/vis_output/blackpink_masked_img.jpg"></td> </tr> <tr> <td width="50%" style="text-align:center;color:gray;"> Please describe the two cameras. <span style="font-weight: bold; color: #FF7F50;">+ DEFAULT_INSTANT_SEG</span> </td> <td width=50% style="text-align:center;">" Sure, [SEG] , [SEG] , [SEG] , and [SEG] ."</td> </tr> </table>

Citaion

@article{yang2023lisa++,
  title={Lisa++: An improved baseline for reasoning segmentation with large language model},
  author={Yang, Senqiao and Qu, Tianyuan and Lai, Xin and Tian, Zhuotao and Peng, Bohao and Liu, Shu and Jia, Jiaya},
  journal={arXiv preprint arXiv:2312.17240},
  year={2023}
}