qwen2.5-3b-oncology-lora

Qwen2.5-3B Oncology Expert (LoRA)

Model Details

Model Description

This is a highly specialized, locally fine-tuned LoRA adapter for the Qwen2.5-3B model, designed specifically for clinical oncology, cancer genetics, and multi-stage treatment regimens. It was trained to act as a domain expert in a Multi-LoRA routing system.

• Developed by: Hriday75
• Model type: LoRA Adapter (PEFT)
• Language: English
• Finetuned from base model: unsloth/Qwen2.5-3B-bnb-4bit
• Training Stage: Phase 1 (Continued Pre-Training on Domain Knowledge)

Uses

This model is intended to be used as a dynamically loaded adapter. When attached to the base Qwen2.5-3B model, it acts as a highly technical oncology expert capable of parsing complex cancer biology and biopsy reports.

Training Details

Training Data

This adapter was trained on high-quality, unstructured medical literature specifically focused on oncology. The primary dataset utilized was cyberpsych/PubMed-Cancer-NLP-Textual-Dataset, alongside open-access medical abstracts focusing on carcinomas, metastatic pathways, and the tumor microenvironment.

Training Procedure

Trained using unsloth for high-efficiency 4-bit LoRA fine-tuning.

How to Get Started with the Model

from peft import PeftModel
from transformers import AutoModelForCausalLM

# 1. Load the base model
base_model = AutoModelForCausalLM.from_pretrained("unsloth/Qwen2.5-3B-bnb-4bit")

# 2. Attach this Oncology Adapter
model = PeftModel.from_pretrained(base_model, "Hriday75/qwen2.5-3b-oncology-lora")