qwen2.5-3b-oncology-chat

Qwen2.5-3B Oncology Chat Assistant (SFT)

Model Details

Model Description

This is a highly specialized, supervised fine-tuned (SFT) LoRA adapter for the Qwen2.5-3B model. While the Phase 1 model learned dense oncology science and cellular biology, this Phase 2 model was specifically instruction-tuned to communicate those facts with excellent bedside manner. It is designed to act as an empathetic, conversational oncology expert.

• Developed by: Hriday75
• Model type: LoRA Adapter (PEFT)
• Language: English
• Finetuned from base model: unsloth/Qwen2.5-3B-bnb-4bit (via Phase 1 Oncology LoRA)
• Training Stage: Phase 2 (Supervised Fine-Tuning / Instruction Tuned)

Uses

This model is intended to be used in a conversational interface. It is designed to answer patient questions about cancer diagnoses, explain complex biopsy reports or chemotherapy regimens simply, and maintain a highly compassionate and helpful tone.

Training Details

Training Data

This adapter was trained on structured conversational datasets featuring thousands of back-and-forth patient/doctor interactions. The primary dataset used was ruslanmv/ai-medical-chatbot, which was formatted into the standard ChatML structure (System, User, Assistant).

Training Procedure

Trained using unsloth and the TRL (Transformer Reinforcement Learning) SFTTrainer for high-efficiency instruction tuning.

How to Get Started with the Model

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

# 1. Load the base model and tokenizer
base_model = AutoModelForCausalLM.from_pretrained("unsloth/Qwen2.5-3B-bnb-4bit")
tokenizer = AutoTokenizer.from_pretrained("unsloth/Qwen2.5-3B-bnb-4bit")

# 2. Attach this Oncology Chat Adapter
model = PeftModel.from_pretrained(base_model, "Hriday75/qwen2.5-3b-oncology-chat")

# 3. Format your chat prompt
messages = [
    {"role": "system", "content": "You are a helpful, empathetic oncology expert."},
    {"role": "user", "content": "Hi doctor, my biopsy results mention 'invasive ductal carcinoma'. What does this mean, and what happens next?"}
]

inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt"
)