Idefic_medical_VQA_merged_4bit

Contributed by:

• Shashwath P
• Shashank Ashok
• Akilan Yohendiran

Total downloads all time - 2106

Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->

The following model is an experimental fine tuned model of the IDEFIC 9B version, for medical Visual Question Answering. It uses a dataset combined from SLAKE and VQARAD. Check the following repository for the notebooks of training,merging and inference. https://github.com/Shashwathp/Idefic_medical_vqa

Model Description

<!-- Provide a longer summary of what this model is. -->

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

• Developed by: [@Shashwath01,@Akill19,@Shashank91097 ]
• Model type: [Multimodal, Visual Question Answering]
• Language(s) (NLP): [English]
• License: [Apache - 2.0]
• Finetuned from model [optional]: [IDEFIC 9B]

Dataset

https://huggingface.co/datasets/Shashwath01/VQARAD_SLAKE

Model Sources

<!-- Provide the basic links for the model. -->

• Repository: https://github.com/Shashwathp/Idefic_medical_vqa

<!--- **Paper :** https://ieeexplore.ieee.org/document/10616779-->

How to Get Started with the Model

Check the below link to get started with inferencing. https://github.com/Shashwathp/Idefic_medical_vqa/blob/main/inference.ipynb

<!--## Citation If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. [1] S. Punneshetty, S. Ashok, M. Niranjanamurthy, and S. V. N. Murthy, "Fine Tuning Idefic 9b With LORA for Multimodal Medical VQA," in *Proceedings of the 2024 International Conference on Knowledge Engineering and Communication Systems (ICKECS)*, India, Apr. 2024, pp. 1-8. DOI: 10.1109/ICKECS61492.2024.10616779.-->