medsentinel-adr-deberta

MedSentinel — ADR Severity Classifier

MedSentinel is a fine-tuned DeBERTa-v3-Base model for classifying the severity of Adverse Drug Reactions (ADRs) from patient-reported narrative text. It is the core AI component of the MedSentinel ADR Intelligence Platform, designed to assist clinical practitioners in triaging pharmacovigilance signals.

Model Details

Intended Use

This model is intended for research and clinical decision support in the context of pharmacovigilance. It classifies free-text patient ADR reports as either severe or non-severe to help clinicians prioritize signals requiring immediate attention.

Intended users: Clinical practitioners, pharmacovigilance researchers, healthcare data scientists.

Out-of-scope uses: This model should not be used as a sole basis for clinical decisions. It is a decision-support tool and should always be reviewed by a qualified healthcare professional.

Training Data

The model was trained on a dataset of 8,153 patient-reported drug experience narratives sourced from drug review platforms. Labels indicate ADR severity:

• 0 — Non-severe adverse drug reaction
• 1 — Severe adverse drug reaction

Class distribution: 53.4% severe · 46.6% non-severe (near-balanced)

Training Configuration

# Key hyperparameters
learning_rate         = 2e-5
optimizer             = "adafactor"
batch_size            = 16  # effective 64 with gradient accumulation
gradient_accumulation = 4
epochs                = 8   # with early stopping (patience=3)
warmup_ratio          = 0.1
lr_scheduler          = "cosine"
weight_decay          = 0.01
max_seq_length        = 256
fp16                  = False  # DeBERTa-v3 incompatibility
cv_folds              = 5

Evaluation Results

How to Use

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
from scipy.special import softmax

model_id  = "Izziemirg/medsentinel-adr-deberta"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model     = AutoModelForSequenceClassification.from_pretrained(model_id)
model.eval()

def classify_adr(text):
    inputs = tokenizer(
        text,
        return_tensors="pt",
        truncation=True,
        max_length=256,
        padding=True
    )
    with torch.no_grad():
        logits = model(**inputs).logits.numpy()

    probs    = softmax(logits, axis=-1)[0]
    label    = "Severe" if probs[1] > 0.5 else "Non-Severe"
    return {"label": label, "confidence": round(float(probs.max()), 4)}

# Example
text = "I experienced severe insomnia, heart palpitations, and extreme anxiety 
        after taking this medication for two weeks."
print(classify_adr(text))
# {'label': 'Severe', 'confidence': 0.9731}

Limitations

• Trained on English-language patient-reported text only
• Performance may degrade on formal clinical notes (different register than training data)
• Mixed-sentiment texts (severe symptoms but positive drug efficacy) remain a known edge case — the model may under-predict severity in these cases
• Not validated on real-world clinical deployment data

Citation

If you use this model in your research, please cite:

@misc{mirghani2025medsentinel,
  title        = {MedSentinel: ADR Severity Classification with DeBERTa-v3},
  author       = {Mirghani, Izzie},
  year         = {2026},
  howpublished = {HuggingFace Hub},
  url          = {https://huggingface.co/Izziemirg/medsentinel-adr-deberta}
}

Developed By

Izzie Mirghani MS Business Analytics, UVA Darden
Part of the MedSentinel ADR Intelligence Platform project.