mistral-medqa

Mistral-MedQA (Medical QA Fine-Tuned Model)

Model Description

This model is a fine-tuned version of the base model:

mistralai/Mistral-7B-v0.1

The model has been fine-tuned on a medical question-answering dataset (MedQA) to improve its ability to answer medical queries and perform domain-specific reasoning.

---

Intended Use

• Medical question answering
• Healthcare-related chatbot systems
• Educational and research purposes

Note: This model is not intended for real-world medical diagnosis without professional supervision.

---

Training Details

Base Model

• mistralai/Mistral-7B-v0.1

Fine-Tuning Method

• Supervised Fine-Tuning (SFT)
• LoRA (Low-Rank Adaptation) using PEFT

---

Dataset

The model was fine-tuned on the MedQA dataset:

https://huggingface.co/datasets/openlifescienceai/medqa

---

Training Hyperparameters

• Epochs: 3
• Learning Rate: 2e-5
• Batch Size: 2
• Gradient Accumulation Steps: 8
• Max Sequence Length: 512
• LoRA Rank (r): 8
• LoRA Alpha: 16
• LoRA Dropout: 0.05

Note: Update these values if they differ from your actual training configuration.

---

Evaluation Results

The model maintains stable general performance and shows baseline domain adaptation. Further improvements are in progress.

---

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "aparnavirtuonai/mistral-medqa-final"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")

prompt = "Question: What is diabetes?\nAnswer:"

inputs = tokenizer(prompt, return_tensors="pt").to("cuda")

outputs = model.generate(**inputs, max_new_tokens=100)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))