distilbert-drug-cls

Drug Review Condition Classifier (DistilBERT)

This model is a multi-class text classification model trained to predict a medical condition based on patient drug reviews from the Drugs.com dataset.

---

📌 Model Overview

• Base model: distilbert-base-uncased
• Task: Text Classification
• Number of labels: ~770+ medical conditions
• Max sequence length: 256
• Training epochs: 3
• Optimizer: AdamW
• Weight decay: 0.01

---

📊 Evaluation Results

Validation

• Accuracy: ~0.74
• Macro F1: ~0.15
• Loss: ~1.15

Test

• Accuracy: ~0.74
• Macro F1: ~0.15
• Loss: ~1.13

Macro F1 is relatively low due to strong class imbalance and a large number of rare condition labels. Accuracy reflects strong performance on frequent condition classes.

---

🧠 Training Details

• Hugging Face Trainer API
• Dynamic padding with DataCollatorWithPadding
• Automatic acceleration via Accelerate
• Train-only label space (no label leakage)
• Evaluation on held-out validation and test splits

---

🚀 Example Usage

from transformers import pipeline

classifier = pipeline(
    "text-classification",
    model="Talip7/distilbert-drug-cls"
)

classifier(
    "This medication significantly reduced my migraine but caused nausea and dizziness."
)

---

⚠️ Limitations

Highly imbalanced class distribution

User-generated reviews may contain noise

Not intended for medical advice or diagnosis

---

📚 Dataset

Source: Drugs.com reviews dataset

Preprocessing:

lowercasing

HTML cleanup

minimum review length filtering

---

👨‍💻 Author

Trained and published as part of hands-on NLP / LLM learning with Hugging Face.