drug-discovery-qwen-14b

Drug Discovery Model - Qwen 2.5 14B Finetuned

This model is a fine-tuned version of Qwen 2.5 14B for drug discovery and molecular property prediction. It predicts whether a drug compound (represented as a SMILES string) is likely to be approved or fail in clinical trials.

Model Description

• Base Model: Qwen/Qwen2.5-14B
• Task: Binary classification (Drug Approval Prediction)
• Training Hardware: AMD MI300X 192GB GPU
• Training Framework: HuggingFace Transformers with 4-bit quantization

Evaluation Results

Training Data

The model was trained on a curated dataset of drug molecules with known clinical trial outcomes:

• Format: SMILES molecular representations
• Labels: Binary (0 = Failed, 1 = Approved)

Training Hyperparameters

• Epochs: 3
• Batch Size: 1 (with gradient accumulation)
• Learning Rate: 1e-5
• Optimizer: AdamW
• Precision: bfloat16 with 4-bit quantization (NF4)
• Gradient Checkpointing: Enabled

How to Use

from transformers import AutoModelForSequenceClassification, AutoTokenizer
import torch

# Load model and tokenizer
model_name = "kprsnt/drug-discovery-qwen-14b"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

# Example SMILES string (Aspirin)
smiles = "CC(=O)OC1=CC=CC=C1C(=O)O"

# Tokenize and predict
inputs = tokenizer(smiles, return_tensors="pt", truncation=True, max_length=512)
with torch.no_grad():
    outputs = model(**inputs)
    probs = torch.softmax(outputs.logits, dim=-1)
    prediction = torch.argmax(probs, dim=-1).item()

print(f"Prediction: {'Approved' if prediction == 1 else 'Failed'}")
print(f"Confidence: {probs[0][prediction].item():.2%}")

Limitations

• Model predictions should not replace expert judgment in drug development
• Performance may vary on molecules very different from training data
• SMILES representation may not capture all relevant molecular properties

Citation

@misc{drug-discovery-qwen-14b,
  author = {Prashanth Kumar},
  title = {Drug Discovery Model - Qwen 2.5 14B Finetuned},
  year = {2024},
  publisher = {HuggingFace},
  url = {https://huggingface.co/kprsnt/drug-discovery-qwen-14b}
}