medical-llama31-8b-sft-lora

AshuReddy/medical-llama31-8b-sft-lora

LoRA-finetuned adapter for Llama-3.1 8B (4-bit) on the FreedomIntelligence/medical-o1-reasoning-SFT (en) dataset.

Overview

• Base model: unsloth/llama-3.1-8b-bnb-4bit
• Method: PEFT / LoRA via Unsloth
• Data: 500 train, 100 validation (10% split from dataset train)
• Prompt template:

You are a careful medical assistant.
Read the question and provide the best final answer.

Question:
{question}

Answer:

Training (key args)

• r=16, lora_alpha=16, lora_dropout=0.05
• epochs=1, per_device_train_batch_size=2, gradient_accumulation_steps=8
• lr=2e-4, scheduler=cosine, warmup_ratio=0.1
• max_seq_length=2048, packing=True, fp16=True

Quick start

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

BASE = "unsloth/llama-3.1-8b-bnb-4bit"
ADAPTER = "AshuReddy/medical-llama31-8b-sft-lora"

tok = AutoTokenizer.from_pretrained(ADAPTER)
m = AutoModelForCausalLM.from_pretrained(
    BASE,
    device_map="auto",
    torch_dtype=torch.float16 if torch.cuda.is_available() else torch.float32,
)
m = PeftModel.from_pretrained(m, ADAPTER)
m.eval()

prompt = "You are a careful medical assistant.\n\nQuestion:\nWhat is PFO?\n\nAnswer:"
inputs = tok(prompt, return_tensors='pt').to(m.device)
with torch.no_grad():
    out = m.generate(**inputs, max_new_tokens=256, do_sample=False, pad_token_id=tok.eos_token_id)
print(tok.decode(out[0], skip_special_tokens=True))

Evaluation (heuristic, 50 val samples)

• Base (adapter OFF): 52%
• Fine-tuned (adapter ON): 54%

Simple exact/synonym/overlap metric; not a clinical benchmark.

Intended use & limitations

• Educational/demo purposes only. Not medical advice.
• Model may produce incorrect or outdated information; verify with clinical sources.

Citation

• Dataset: FreedomIntelligence/medical-o1-reasoning-SFT
• Unsloth: https://github.com/unslothai/unsloth