Downloads · 30 days
54
0% of all-time downloads
clinical-pubmed-bert-base-512
Tsubasaz/clinical-pubmed-bert-base-512
clinical-pubmed-bert-base-512 is a fill-mask model from Tsubasaz. Use it when you need the model to fill a missing word. It is set up for transformers. The card lists the license as mit.
A pre-trained model for clinical decision support, for more details, please see https://github.com/NtaylorOX/PublicPromptMimicIII
All-time downloads
17.5K
Public
Repo size
876 MB
Likes
5
Public
Repo makeup
Click a slice to open those files.
.bin438 MB · 100%
At a glance
- Task
- Fill-Mask
- Library
- transformers
- License
- mit
- Model type
- bert
- Access
- Public
- Created
- Mar 2, 2022
- Updated
- May 6, 2022
- SHA
- e5a64340
Try a prompt
Datasets
Model card
From the Hugging Face model README
ClinicalPubMedBERT
Description
A pre-trained model for clinical decision support, for more details, please see https://github.com/NtaylorOX/Public_Prompt_Mimic_III
A BERT model pre-trained on PubMed abstracts, and continual pre-trained on clinical notes (MIMIC-III). We try combining two domains that have fewer overlaps with general knowledge text corpora: EHRs and biomedical papers. We hope this model can serve better results on clinical-related downstream tasks such as readmissions.
This model is trained on 500000 clinical notes randomly sampled from MIMIC datasets, with 100k steps of training. We also used whole word masking to enhance the coherence of the language model. All notes are chunked into a length of 512 tokens.
Pre-trained model: https://huggingface.co/microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract
- Task
- Fill-Mask
- Library
- transformers
- Type
- bert
- License
- mit
- Languages
- en
- Created
- Mar 2, 2022
- Updated
- May 6, 2022