clinical-llm-r32

Clinical LLM — QLoRA Fine-Tune on MedQA (Llama 3.2 3B)

A QLoRA fine-tune of Meta's Llama 3.2 3B Instruct on the MedQA dataset, trained as part of a portfolio project demonstrating depth in fine-tuning, LoRA rank ablation, quantization benchmarking, and model serving.

---

Model Details

• Base model: meta-llama/Llama-3.2-3B-Instruct
• Fine-tuning method: QLoRA via Unsloth + PEFT + TRL
• Dataset: medalpaca/medical_meadow_medqa (2,500 samples — 2,250 train / 250 val)
• Task: Multiple-choice clinical question answering (MedQA / USMLE-style)
• Hardware: Google Colab L4 GPU (23.7 GB VRAM)
• Unsloth version: 2026.3.4 | Transformers: 5.2.0

---

Training Configuration

---

LoRA Rank Ablation Study

All runs trained on identical data and hyperparameters, varying only rank and lora_alpha (convention: lora_alpha = 2 × r).

Key finding: r=32 fine-tune is ~6x more accurate than the base model. The base model's low score is largely a format compliance issue — it generates verbose explanations rather than the concise single-letter answer the benchmark expects.

Best checkpoint: r=32 → used for deployment.

---

Quantization Benchmark (merged r=32, L4 GPU)

4-bit is 27% faster with negligible accuracy loss — strong production argument for quantization.

---

Inference

from unsloth import FastLanguageModel
import torch

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="aakthepaak/clinical-llm-r32",
    max_seq_length=2048,
    load_in_4bit=True,
)
FastLanguageModel.for_inference(model)

prompt = """<|system|> You are a clinical medical assistant, answer the following question.
<|user|> {instruction}\n{question}
<|assistant|>"""

inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=30)
response = tokenizer.decode(outputs[0][inputs['input_ids'].shape[1]:], skip_special_tokens=True)
print(response)

---

License

This model is a fine-tune of Meta's Llama 3.2 3B Instruct and is subject to the Llama 3.2 Community License. Use is permitted for research and commercial purposes under those terms.

---

Links

• WandB experiment tracking: https://wandb.ai/aakarsh3110-stevens-institute-of-technology/clinical-llm
• Base model: https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct
• Dataset: https://huggingface.co/datasets/medalpaca/medical_meadow_medqa