Spatial-LLaVA-7B

<br> <br>

Spatial-LLaVA-7B Model Card

Github Repo

🤗 Huggingface Space Demo

🤖 Model details

Model type:

This finetuned LLaVA model is trained from liuhaotian/llava-pretrain-vicuna-7b-v1.3 for improving spatial relation reasoning of large multi-modal model.

LLaVA is an open-source chatbot trained by fine-tuning LLaMA/Vicuna on GPT-generated multimodal instruction-following data. It is an auto-regressive language model, based on the transformer architecture.

🎯 Intended use

Primary intended uses: The primary use of LLaVA is research on large multimodal models and chatbots.

Primary intended users: The primary intended users of the model are researchers and hobbyists in computer vision, natural language processing, machine learning, and artificial intelligence.

📚 Training dataset

Instruction following training: rogerxi/LLaVA-Spatial-Instruct-850K

📊 Evaluation

A collection of 10 benchmarks:

Spatial-Relation-Eval (built based on SpatialRGPT-Bench):

Qualitative Spatial Relations

Quantitative Spatial Relations

🙏 Acknowledgements

We thank Liu Haotian et al. for the LLaVA pretrained script, weights and LLaVA-v1.5 mixture dataset; the teams behind CLEVR, TextCaps, VisualMRC and VQAv2 (via “HuggingFaceM4/the_cauldron”); remyxai for OpenSpaces; Anjie Cheng et al. for Spatial-Bench and data pipeline; Google for OpenImages; and Hugging Face for their datasets infrastructure.