drug-review-sentiment-analysis-bert

Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->

🧠 Model Card

Drug Review Sentiment Classification (3-Class)

Fine-tuned BERT on Healthcare User Reviews

---

📌 Model Overview

This model is a fine-tuned version of BERT-base-uncased for three-class sentiment classification on user-generated drug reviews.

The task consists of classifying textual reviews into:

• Negative
• Neutral
• Positive

The model was trained on a cleaned and preprocessed version of the Drugs.com review dataset.

---

📂 Dataset

Source: Drugs.com Drug Review Dataset

• User reviews
• Numerical rating (1–10 scale)

Label Mapping Strategy

Three-class sentiment labels were constructed as:

• Negative → Low ratings
• Neutral → Mid-range ratings
• Positive → High ratings

Text preprocessing pipeline:

• Lowercasing
• Punctuation removal
• Stopword filtering
• Lemmatization
• Tokenization using BERT tokenizer

---

🏗 Model Architecture

Base Model: BERT-base-uncased

Architecture Details:

• 12 Transformer encoder layers
• Hidden size: 768
• 12 attention heads
• Classification head added for 3 output labels

Loss Function:

• CrossEntropyLoss

Optimizer:

• AdamW

---

Training performed using:

• PyTorch
• HuggingFace Transformers

---

📊 Evaluation Metrics

Evaluation was conducted on a held-out validation set.

Metrics reported:

• Accuracy
• Precision (macro)
• Recall (macro)
• F1-score (macro)
• Confusion Matrix

Example

| Model | Feature | Accuracy | F1 |

| Logistic Regression | TF-IDF | 0.82 | 0.81 |

| Naive Bayes | TF-IDF | 0.78 | 0.77 |

| Random Forest | TF-IDF | 0.81 | 0.79 |

| LSTM | Word2Vec | 0.83 | 0.83 |

---

🔍 Key Findings

• Transformer-based contextual embeddings outperform classical TF-IDF based models in multi-class sentiment detection.
• The neutral class benefits significantly from contextual modeling.
• Domain-specific vocabulary (medical terminology) is better captured by BERT representations.

---

🧪 Example Inference

from transformers import pipeline

classifier = pipeline(
    "text-classification",
    model="erdemyavuz55/drug-review-sentiment-analysis-bert"
)

classifier("This medication worked great with minimal side effects.")

Example Output:

[{'label': 'POSITIVE', 'score': 0.97}]

---

📈 Comparison with Classical Models

In the broader study, we compared:

• TF-IDF + Logistic Regression
• TF-IDF + SVM
• Word2Vec + LSTM
• Random Forest
• Gradient Boosting

The fine-tuned BERT model achieved superior macro-F1 in the three-class setting.

---

⚠️ Limitations

• Trained only on English reviews
• Domain-specific slang may reduce accuracy
• Ratings-based labeling may introduce subjectivity bias
• No domain-adaptive pretraining performed

---

🚀 Future Work

• Domain-adaptive pretraining on medical corpora
• Model interpretability (SHAP, attention visualization)
• Deployment as REST API
• Cross-dataset generalization experiments

---

🤗 Citation

If you use this model, please cite:

Fine-tuned BERT for Drug Review Sentiment Classification, 2026.