Downloads · 30 days
21
20% of all-time downloads
d1wash/mistral-7b-healthcare-qlora
mistral-7b-healthcare-qlora is a machine learning model from d1wash. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as apache-2.0.
A QLoRA (4-bit NF4 quantization + LoRA) fine-tune of mistralai/Mistral-7B-Instruct-v0.2 on the ChatDoctor-HealthCareMagic-100k dataset, adapting the base instruction-tuned model to respond in the style of a medical Q&…
Downloads · 30 days
21
20% of all-time downloads
All-time downloads
107
Public
Repo size
13.6 MB
Likes
1
Public
Click a slice to open those files.
.safetensors13.6 MB · 80%
From the Hugging Face model README
A QLoRA (4-bit NF4 quantization + LoRA) fine-tune of mistralai/Mistral-7B-Instruct-v0.2 on the
ChatDoctor-HealthCareMagic-100k dataset, adapting the base instruction-tuned model to respond in the
style of a medical Q&A assistant.
⚠️ Disclaimer: This is a research/portfolio project. It is not a substitute for professional medical advice, diagnosis, or treatment. Do not use this model for real medical decision-making. Always consult a qualified healthcare provider for medical concerns.
This adapter is intended to be loaded on top of the Mistral-7B-Instruct-v0.2 base model to generate conversational responses in a medical Q&A style, for research, demonstration, and educational purposes.
Users should treat all outputs as illustrative only, verify any medical claims with a qualified professional, and not rely on this model for real health decisions.
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
base_model_name = "mistralai/Mistral-7B-Instruct-v0.2"
adapter_repo = "d1wash/mistral-7b-healthcare-qlora"
tokenizer = AutoTokenizer.from_pretrained(base_model_name)
tokenizer.pad_token = tokenizer.eos_token
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=torch.float16,
bnb_4bit_use_double_quant=True,
)
base_model = AutoModelForCausalLM.from_pretrained(
base_model_name,
quantization_config=bnb_config,
device_map="auto",
)
model = PeftModel.from_pretrained(base_model, adapter_repo)
prompt = """<s>[INST] You are a helpful medical assistant.
I have been experiencing persistent headaches for the past week. What could be the cause?
Patient: I wake up with a throbbing headache every morning. [/INST]
Doctor:"""
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
output = model.generate(
**inputs,
max_new_tokens=200,
do_sample=True,
temperature=0.7,
top_p=0.9,
repetition_penalty=1.3,
no_repeat_ngram_size=3,
pad_token_id=tokenizer.eos_token_id,
)
print(tokenizer.decode(output[0], skip_special_tokens=True))
A filtered, subsampled subset of ChatDoctor-HealthCareMagic-100k:
<s>[INST] ... [/INST] ...q_proj, v_projTraining and validation loss decreased consistently across checkpoints:
| Step | Training Loss | Validation Loss |
|---|---|---|
| 150 | 2.004 | 2.011 |
| 300 | 1.900 | 1.907 |
| 450 | 1.880 | 1.885 |
| 600 | 1.903 | 1.870 |
Validation loss tracked closely with training loss throughout, indicating the model generalized to the held-out set rather than memorizing the training data. Training completed the full 900-step run with the best checkpoint (by evaluation loss) restored at the end.
NVIDIA T4 GPU (16GB VRAM), 4 vCPUs — Lightning AI Studio free tier
transformers, peft, bitsandbytes, accelerate, trlDiwash Adhikari — d1wash on Hugging Face | GitHub: Diwash17