mult_tf

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

mult_tf

This model is a fine-tuned version of microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract-fulltext on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.5180
• Accuracy: 0.8364
• F1: 0.8358
• Precision: 0.8355
• Recall: 0.8364
• Roc Auc: 0.9896

Model description

mult_tf is a fine-tuned [PubMedBERT](https://huggingface.co/microsoft/BiomedNLP-PubMedBERT-base -uncased-abstract-fulltext) model for 17-class medical specialty classification of biomedical text.
It distinguishes between 11 Internal Medicine sub-specialties and 6 other medical disciplines, trained on 300,000 PubMed article titles using journal provenance as a distant supervision
signal. No manual annotation was used.

Companion to the binary classifier
tgamstaetter/im-bin-tf-abstr.

Intended uses & limitations

• Fine-grained specialty classification of biomedical abstracts or titles

  • Research on multiclass distant supervision in biomedical NLP

  Not intended for: clinical decision support, diagnostic use, or any safety-critical application.

Training and evaluation data

300,000 PubMed article titles from 77 medical journals. Labels from journal editorial scope
(distant supervision).

17 classes:

Dataset: Internal medicine and other specialties — Kaggle

Training procedure

Fine-tuned from microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract-fulltext:

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 640
• eval_batch_size: 1280
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 4

Training results

Evaluation results

Evaluated on a held-out test set of 100,000 titles:

Lowest per-class F1 scores: intensive care medicine (0.670), geriatrics (0.683),
angiology (0.704) — reflecting known clinical content overlap with adjacent specialties.

Limitations

• Trained on article titles only.
• Label noise at disciplinary boundaries (e.g., intensive care / anesthesiology, angiology / cardiology) is inherent to the distant supervision approach.
• Evaluated on English-language text only.

Framework versions

• Transformers 4.31.0

• Pytorch 2.0.1+cu118

• Datasets 2.13.1

• Tokenizers 0.13.3

  Citation

  @misc{gamstaetter2023modelmc,
    author       = {Gamstaetter, Thomas},                                                          
    title        = {mult\_tf: Fine-tuned {PubMedBERT} for multiclass medical specialty
  classification},                                                                                 
    year         = {2023},
    howpublished = {Hugging Face},                                                                 
    url          = {https://huggingface.co/tgamstaetter/mult_tf}
  }                                                                                                
  

  Associated preregistration: OSF — DOI 10.17605/OSF.IO/XFDBV