drug-discovery-chemberta

Drug Discovery Model - ChemBERTa Finetuned

A ChemBERTa model finetuned for predicting drug approval success based on molecular SMILES representation.

Model Description

This model is a finetuned version of seyonec/ChemBERTa-zinc-base-v1 for drug success prediction. It classifies molecules as likely to be approved or failed based on their SMILES structure.

Training Details

• Base Model: ChemBERTa-zinc-base-v1 (~85M parameters)
• Training: Full finetuning (not LoRA)
• Hardware: NVIDIA RTX 3050 6GB
• Optimization: Gradient checkpointing + FP16 mixed precision
• Dataset: ChEMBL, DrugBank, FDA drugs (approved + withdrawn)

Performance

Comparison with Pretrained

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

# Load model and tokenizer
tokenizer = AutoTokenizer.from_pretrained("YOUR_USERNAME/drug-discovery-chemberta")
model = AutoModelForSequenceClassification.from_pretrained("YOUR_USERNAME/drug-discovery-chemberta")

# Predict for a molecule (Aspirin)
smiles = "CC(=O)OC1=CC=CC=C1C(=O)O"
inputs = tokenizer(smiles, return_tensors="pt", padding=True, truncation=True, max_length=128)

with torch.no_grad():
    outputs = model(**inputs)
    probs = torch.softmax(outputs.logits, dim=-1)
    prediction = "Approved" if probs[0][1] > 0.5 else "Failed"
    confidence = probs[0][1].item() if prediction == "Approved" else probs[0][0].item()

print(f"Prediction: {prediction} (confidence: {confidence:.2%})")

Labels

• 0: Failed/Withdrawn drug
• 1: Approved drug

Intended Use

This model is intended for:

• Drug discovery research
• Virtual screening of candidate molecules
• Prioritizing compounds for further development

Limitations

• Trained on small molecule drugs only
• Predictions are probabilistic and should not replace clinical trials
• Performance may vary for novel chemical scaffolds

Citation

If you use this model, please cite:

@misc{drug-discovery-chemberta-2024,
  author = {Your Name},
  title = {Drug Discovery ChemBERTa: Finetuned for Drug Success Prediction},
  year = {2024},
  publisher = {Hugging Face},
  url = {https://huggingface.co/YOUR_USERNAME/drug-discovery-chemberta}
}

License

MIT License