MedLLaMA3-DDI-QLoRA-V1

🧬 MedLLaMA3-DDI-QLoRA-V1

PharmaMap-LLM: A QLoRA Fine-Tuned MedLLaMA-3 Model for Drug-Drug Interaction (DDI) Prediction

This model is part of the MSc research project PharmaMap-LLM, developed to enable memory-efficient biomedical relation extraction using instruction-tuned Large Language Models (LLMs). It fine-tunes johnsnowlabs/JSL-MedLlama-3-8B-v2.0 using QLoRA on DrugBank-based datasets for DDI classification and reasoning.

---

📌 Model Summary

• Base Model: johnsnowlabs/JSL-MedLlama-3-8B-v2.0
• Tuning Method: QLoRA (4-bit quantization + LoRA adapters)
• Task: Drug-drug interaction classification + explanation
• Frameworks: Hugging Face Transformers, PEFT, Datasets
• Prompt Format: Instruction-style (Vicuna-like) with reasoning support
• Instruction Accuracy: 91%
• Memory Reduction: ~60% compared to full-parameter fine-tuning

---

🧠 Example Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

# Load base + LoRA adapter
base_model = AutoModelForCausalLM.from_pretrained("johnsnowlabs/JSL-MedLlama-3-8B-v2.0", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("Pharmamapllm/MedLLaMA3-DDI-QLoRA-V1")
model = PeftModel.from_pretrained(base_model, "Pharmamapllm/MedLLaMA3-DDI-QLoRA-V1")

prompt = """Drug1: Ranolazine
SMILES for drug1: COC1=CC=CC=C1OCC(O)CN1CCN(CC(=O)NC2=C(C)C=CC=C2C)CC1
Organism targeted by drug1: Unknown
Genes targeted by drug1: P00747, P35498, Q14500, Q13936, P35348, P08588
Drug2: Capreomycin
SMILES for drug2: ...
Organism targeted by drug2: Unknown
Genes targeted by drug2: P9WJ63

CLASSIFICATION:"""

inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=150)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

---

📊 Performance

---

🧬 Data & Labeling

• Positive Samples: DrugBank annotated interactions
• Negative Samples: Random drug pairs with no known interactions
• Enhancement: Some positive samples injected with reasoning generated by Arcee.ai
• Schema:
  • Drug names
  • SMILES structures
  • Organism targets
  • Gene targets
• Prompt-based format: "CLASSIFICATION:" triggers generation

---

🧪 Training Configuration

---

📁 Files Included

• adapter_model.safetensors — LoRA adapter weights
• adapter_config.json — PEFT adapter configuration
• tokenizer.model, tokenizer_config.json — Tokenizer files
• special_tokens_map.json — Special token mapping
• README.md — This file!

---

📄 Citation

@misc{raveendran2025pharmamap,
  title={PharmaMap-LLM: Fine-tuning Large Language Models for Drug-Drug Interaction (DDI) Analysis},
  author={Raveendran, Kithusshand},
  year={2025},
  institution={University of Westminster, Informatics Institute of Technology},
  howpublished={\url{https://huggingface.co/Pharmamapllm/MedLLaMA3-DDI-QLoRA-V1}}
}

---

🙋‍♂️ Author

Kithusshand Raveendran
MSc in Advanced Software Engineering
University of Westminster / Informatics Institute of Technology
🔗 LinkedIn Profile

---

📢 Disclaimer

This model is for research and educational purposes only. It is not approved for clinical use. Always consult licensed medical professionals before using automated predictions for any real-world treatment.

---

🧠 Acknowledgements

• johnsnowlabs/JSL-MedLlama-3-8B-v2.0
• DrugBank & DDIExtraction 2013
• Hugging Face Transformers & PEFT
• Arcee.ai for reasoning generation
• University of Westminster & IIT Sri Lanka