biomistral_medqa_v1

Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->

Finetuned "BioMistral/BioMistral-7B" with MedQA dataset.

Model Details

A Collection of Open-Source Pretrained Large Language Models for Medical Domains finetuned with MedQA dataset.

Model Description

<!-- Provide a longer summary of what this model is. -->

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

• Developed by: mychen76
• Model type: BioMedical
• Finetuned from model: BioMistral/BioMistral-7B

Model Sources [optional]

<!-- Provide the basic links for the model. -->

• dataset: MedQA dataset

How to Get Started with the Model

Use the code below to get started with the model.

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

Load Model:

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig

base_model_id = "mychen76/biomistral_medqa_v1"
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16
)

model = AutoModelForCausalLM.from_pretrained(base_model_id, quantization_config=bnb_config)
tokenizer = AutoTokenizer.from_pretrained(
    base_model_id,
    add_eos_token=True,
    add_bos_token=True,
)

## Uses

*** Information ***

eval_prompt = """From the MedQuad MedicalQA Dataset: Given the following medical question and question type, provide an accurate answer:

### Question type:
information

### Question:
What are the genetic changes related to X-linked lymphoproliferative disease ?

### Answer:
"""

model_input = eval_tokenizer(eval_prompt, return_tensors="pt").to("cuda")

ft_model.eval()
with torch.no_grad():
    print(eval_tokenizer.decode(ft_model.generate(**model_input, max_new_tokens=300)[0], skip_special_tokens=True))

result:

From the MedQuad MedicalQA Dataset: Given the following medical question and question type, provide an accurate answer:

### Question type:
information

### Question:
What are the genetic changes related to X-linked lymphoproliferative disease ?

### Answer:
X-linked lymphoproliferative disease (XLP) is a rare primary immunodeficiency syndrome. XLP is caused by mutations in SH2D1A gene, which encodes the cytoplasmic signaling protein SLAM-associated protein ( client protein-SLAM). SLAM is a member of the signaling lymphocytic activation molecule family of receptors, which are involved in the regulation of lymphocyte activation and proliferation. The SLAM receptor is expressed on the surface of B and T lymphocytes, natural killer cells, and monocytes. Mutations in SH2D1A gene lead to impaired signaling through the SLAM receptor, resulting in a deficiency in the activation and proliferation of B and T lymphocytes. This leads to a decrease in the number of B and T lymphocytes, resulting in a weakened immune response.

*** Frequency ***

eval_prompt = """From the MedQuad MedicalQA Dataset: Given the following medical question and question type, provide an accurate answer:

### Question type:
frequency

### Question:
How many people are affected by Smith-Lemli-Opitz syndrome ?

### Answer:
"""

model_input = eval_tokenizer(eval_prompt, return_tensors="pt").to("cuda")

ft_model.eval()
with torch.no_grad():
    print(eval_tokenizer.decode(ft_model.generate(**model_input, max_new_tokens=300)[0], skip_special_tokens=True))

result:

From the MedQuad MedicalQA Dataset: Given the following medical question and question type, provide an accurate answer:

### Question type:
frequency

### Question:
How many people are affected by Smith-Lemli-Opitz syndrome ?

### Answer:
Smith-Lemli-Opitz syndrome (SLOS) is a rare autosomal recessive disorder of human development. It is characterized by a wide range of symptoms, including growth and developmental delay, intellectual disability, characteristic facial features, and congenital heart defects. The prevalence of SLOS is estimated to be 1 in 15,000 to 1 in 25,000 live births.

*** Symptons ***

eval_prompt = """From the MedQuad MedicalQA Dataset: Given the following medical question and question type, provide an accurate answer:

### Question type:
symptoms

### Question:
What are the symptoms of Norrie disease ?

### Answer:
"""

model_input = eval_tokenizer(eval_prompt, return_tensors="pt").to("cuda")

ft_model.eval()
with torch.no_grad():
    print(eval_tokenizer.decode(ft_model.generate(**model_input, max_new_tokens=300)[0], skip_special_tokens=True))

Result:

Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.

From the MedQuad MedicalQA Dataset: Given the following medical question and question type, provide an accurate answer:

### Question type:
symptoms

### Question:
What are the symptoms of Norrie disease ?

### Answer:
Norrie disease is a rare, X-linked recessive disorder of the blood vessels. It is characterized by a variety of symptoms, including glaucoma, mental retardation, seizures, and deafness.

Out-of-Scope Use

images

<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->

[More Information Needed]

Bias, Risks, and Limitations

<!-- This section is meant to convey both technical and sociotechnical limitations. -->

[More Information Needed]

Recommendations

<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

[More Information Needed]

Training Details

Training Data

• dataset: keivalya/MedQuad-MedicalQnADataset

<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

[More Information Needed]

Training Procedure

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->

Citation

Arxiv : https://arxiv.org/abs/2402.10373

@misc{labrak2024biomistral, title={BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domains}, author={Yanis Labrak and Adrien Bazoge and Emmanuel Morin and Pierre-Antoine Gourraud and Mickael Rouvier and Richard Dufour}, year={2024}, eprint={2402.10373}, archivePrefix={arXiv}, primaryClass={cs.CL} }