medical_llama3

Medical LLaMA 3.1 8B (QLoRA, rsLoRA)

LLaMA 3.1 8B Instruct fine-tuned for medical question answering, built for MedAssist: a cited, guarded medical information assistant (B.Tech project, Group 36, Symbiosis Institute of Technology, Pune).

Code: github.com/sourav-singh-10/medical-llm-safety-guardrails Training data: sourxv-17/llama3.1-medical-dataset Routing benchmark: sourxv-17/medroute-bench

Files

Training

• QLoRA: base weights frozen in 4-bit NF4; rank-stabilised LoRA adapters (r = 32, α = 32) on q, k, v, o, gate, up, down (83.9 M trainable parameters, 1.03%)
• 59,612 curated medical QA rows (ChatDoctor-HealthCareMagic, MedMCQA explanations, PubMedQA) + 1,050 guardrail rows; loss on answer tokens only
• Batch 2 × 8 accumulation, AdamW 8-bit, LR 2e-4 with warm-up and cosine decay, one T4 GPU
• Selected checkpoint-3400 (lowest validation loss, 1.767); validation loss rose afterwards

Evaluation (held-out test data, Tesla T4)

Fine-tuning weakened off-topic refusal, so MedAssist enforces refusal with its own topic router and guardrails rather than relying on the model.

Use with Ollama

# Modelfile
FROM ./medical-llama3.1-8b-Q4_K_M.gguf
TEMPLATE """{{ range .Messages }}<|start_header_id|>{{ .Role }}<|end_header_id|>

{{ .Content }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>

"""
PARAMETER stop "<|eot_id|>"
PARAMETER temperature 0.2

ollama create medical-llama3.1 -f Modelfile
ollama run medical-llama3.1 "What are the symptoms of dengue?"

Limitations

For education and health information only. It is not a doctor, does not diagnose, and can be wrong. In an emergency in India call 108 or 112; for mental-health support call Tele-MANAS 14416.