Downloads · 30 days
16
4% of all-time downloads
Francesco-A/BiomedNLP-PubMedBERT-base-uncased-abstract-bc5cdr-ner-LoRA-v1
BiomedNLP-PubMedBERT-base-uncased-abstract-bc5cdr-ner-LoRA-v1 is a token classification model from Francesco-A. Use it when you need labels on individual words, such as names. It is set up for peft. The card lists the license as mit.
This is a LoRA fine-tuned adapter based on microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract. It has been fine-tuned on the BC5CDR dataset, which focuses on biomedical named entity recognition (NER) for chemicals a…
Downloads · 30 days
16
4% of all-time downloads
All-time downloads
382
Public
Repo size
77.4 MB
Likes
0
Public
Click a slice to open those files.
.safetensors5.2 MB · 85%
From the Hugging Face model README
This is a LoRA fine-tuned adapter based on microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract. It has been fine-tuned on the BC5CDR dataset, which focuses on biomedical named entity recognition (NER) for chemicals and diseases. The adapter is designed to enhance PubMedBERT’s performance on biomedical text-mining tasks by improving its ability to recognize and classify domain-specific entities while maintaining computational efficiency. It achieves the following results on the evaluation set:
⚠️ Note: This model is shared for educational and testing purposes only. It is not intended for clinical or production use and may not generalize beyond the BC5CDR dataset.
ChemicalDiseaseO (non-entity tokens)The model predicts entity spans following the standard BIO tagging scheme, e.g. B-Chemical, I-Disease, O.
The full training and preprocessing pipeline — including dataset preparation, sentence splitting, BIO tagging, and evaluation — is documented in this Kaggle notebook
from peft import AutoPeftModelForTokenClassification
from transformers import AutoTokenizer, AutoModelForTokenClassification
from transformers import pipeline
# Define labels
LABEL_NAMES = ["O", "B-Chemical", "I-Chemical", "B-Disease", "I-Disease"]
id2label = {i: label for i, label in enumerate(LABEL_NAMES)}
label2id = {label: i for i, label in enumerate(LABEL_NAMES)}
tokenizer = AutoTokenizer.from_pretrained("Francesco-A/BiomedNLP-PubMedBERT-base-uncased-abstract-bc5cdr-ner-LoRA-v1")
model = AutoPeftModelForTokenClassification.from_pretrained(
"Francesco-A/BiomedNLP-PubMedBERT-base-uncased-abstract-bc5cdr-ner-LoRA-v1",
num_labels=len(id2label),
id2label=id2label,
label2id=label2id,
)
ner = pipeline("token-classification", model=model, tokenizer=tokenizer, aggregation_strategy="first")
demo_texts = [
"Aspirin is often used to treat inflammation, but may cause gastric bleeding.",
"Naloxone reverses the antihypertensive effect of clonidine.",
]
for t in demo_texts:
print("\nText:", t)
preds = ner(t)
for p in preds:
print(f' - {p["word"]}: {p["entity_group"]} (score {p["score"]:.2f})')
The following hyperparameters were used during training:
| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1 | Accuracy |
|---|---|---|---|---|---|---|---|
| 0.6498 | 1.0 | 121 | 0.1174 | 0.8015 | 0.8380 | 0.8193 | 0.9617 |
| 0.1057 | 2.0 | 242 | 0.0888 | 0.8663 | 0.8721 | 0.8692 | 0.9719 |
| 0.0747 | 3.0 | 363 | 0.0844 | 0.8568 | 0.8942 | 0.8751 | 0.9707 |
| 0.0624 | 4.0 | 484 | 0.0802 | 0.8802 | 0.8812 | 0.8807 | 0.9729 |
| 0.0486 | 5.0 | 605 | 0.0930 | 0.8575 | 0.9037 | 0.88 | 0.9704 |
| 0.0412 | 6.0 | 726 | 0.0961 | 0.8770 | 0.9019 | 0.8893 | 0.9728 |
| 0.0324 | 7.0 | 847 | 0.0895 | 0.8984 | 0.8909 | 0.8947 | 0.9752 |
| 0.0274 | 8.0 | 968 | 0.1082 | 0.8856 | 0.9015 | 0.8935 | 0.9750 |
| 0.0209 | 9.0 | 1089 | 0.1278 | 0.8969 | 0.8976 | 0.8972 | 0.9749 |
| 0.0176 | 10.0 | 1210 | 0.1182 | 0.8891 | 0.9007 | 0.8948 | 0.9733 |
| 0.0153 | 11.0 | 1331 | 0.1406 | 0.8986 | 0.8983 | 0.8984 | 0.9751 |
| 0.0134 | 12.0 | 1452 | 0.1413 | 0.8909 | 0.9014 | 0.8961 | 0.9745 |
| 0.0111 | 13.0 | 1573 | 0.1423 | 0.8909 | 0.9094 | 0.9000 | 0.9755 |
| 0.0088 | 14.0 | 1694 | 0.1519 | 0.8927 | 0.9052 | 0.8989 | 0.9748 |
| 0.0069 | 15.0 | 1815 | 0.1720 | 0.8942 | 0.9025 | 0.8984 | 0.9754 |