qwen2.5-3b-infectious-disease-lora

Qwen2.5-3B Infectious Disease Expert (LoRA)

Model Details

Model Description

This is a highly specialized, locally fine-tuned LoRA adapter for the Qwen2.5-3B model, designed specifically for infectious diseases, virology, and epidemiology. It was trained to act as a domain expert in a Multi-LoRA routing system.

• Developed by: Hriday75
• Model type: LoRA Adapter (PEFT)
• Language: English
• Finetuned from base model: unsloth/Qwen2.5-3B-bnb-4bit
• Training Stage: Phase 1 (Continued Pre-Training on Domain Knowledge)

Uses

This model is intended to be used as a dynamically loaded adapter. When attached to the base Qwen2.5-3B model, it acts as a highly technical infectious disease expert.

Training Details

Training Data

This adapter was trained on high-quality, unstructured medical literature specifically focused on infectious diseases. The primary dataset used was KushT/LitCovid_BioCreative, which contains tens of thousands of parsed PubMed abstracts, virology papers, and epidemiology reports.

Training Procedure

Trained using unsloth for high-efficiency 4-bit LoRA fine-tuning.

How to Get Started with the Model

from peft import PeftModel
from transformers import AutoModelForCausalLM

# 1. Load the base model
base_model = AutoModelForCausalLM.from_pretrained("unsloth/Qwen2.5-3B-bnb-4bit")

# 2. Attach this Infectious Disease Adapter
model = PeftModel.from_pretrained(base_model, "Hriday75/qwen2.5-3b-infectious-disease-lora")