Downloads · 30 days
15
6% of all-time downloads
distilbert-medical-classifier
AbdoMatrix/distilbert-medical-classifier
distilbert-medical-classifier is a text classification model from AbdoMatrix. Use it when you need a label for a piece of text. The card lists the license as mit.
Fine-tuned distilbert-base-uncased for classifying medical questions into 6 categories.
All-time downloads
250
Public
Parameters
67M
3.2 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors268 MB · 100%
At a glance
- Task
- Text Classification
- License
- mit
- Model type
- distilbert
- Access
- Public
- Created
- Apr 29, 2026
- Updated
- May 18, 2026
- SHA
- bc494223
Try a prompt
Datasets
Model card
From the Hugging Face model README
DistilBERT Medical Query Classifier
Fine-tuned distilbert-base-uncased for classifying medical questions into 6 categories.
Categories
| ID | Category |
|---|---|
| 0 | Diagnosis |
| 1 | General |
| 2 | Medication |
| 3 | Prevention |
| 4 | Symptoms |
| 5 | Treatment |
Results
| Metric | Score |
|---|---|
| Macro F1 | 0.8670 |
| Weighted F1 | 0.8787 |
| Accuracy | 0.8790 |
Training Config
| Item | Value |
|---|---|
| Base model | distilbert-base-uncased |
| Dataset | llamafactory/PubMedQA (10,000 rows) |
| Split | 80/10/10 |
| Epochs | 3 |
| Learning rate | 2e-5 |
| Batch size | 16 |
| Class weights | Balanced (custom WeightedTrainer) |
Usage
from transformers import DistilBertTokenizer, DistilBertForSequenceClassification import torch
tokenizer = DistilBertTokenizer.from_pretrained("AbdoMatrix/distilbert-medical-classifier") model = DistilBertForSequenceClassification.from_pretrained("AbdoMatrix/distilbert-medical-classifier")
text = "What are the symptoms of diabetes?" inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=256)
with torch.no_grad(): outputs = model(**inputs)
predicted = model.config.id2label[torch.argmax(outputs.logits, dim=1).item()] print(predicted) # → Symptoms
Project
Healthcare RAG-Powered Medical Q&A Assistant eyouth x DEPI | Microsoft Machine Learning Track | 2026 GitHub: https://github.com/AbdooMatrix/Healthcare-RAG-Powered-Medical-QA-Assistant
- Task
- Text Classification
- Type
- distilbert
- License
- mit
- Languages
- en
- Created
- Apr 29, 2026
- Updated
- May 18, 2026