Downloads · 30 days
7K
7% of all-time downloads
efficient_mlm_m0.40
andreasmadsen/efficient_mlm_m0.40
efficient_mlm_m0.40 is a fill-mask model from andreasmadsen. Use it when you need the model to fill a missing word. It is set up for transformers.
This is a model checkpoint for "Should You Mask 15% in Masked Language Modeling" (code).
All-time downloads
95.5K
Public
Repo size
4.5 GB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.h51.6 GB · 53%
At a glance
- Task
- Fill-Mask
- Library
- transformers
- Model type
- roberta-prelayernorm
- Access
- Public
- Created
- Nov 15, 2022
- Updated
- Nov 15, 2022
- SHA
- 3924d3fe
Try a prompt
Model card
From the Hugging Face model README
This is a model checkpoint for "Should You Mask 15% in Masked Language Modeling" (code).
The original checkpoint is avaliable at princeton-nlp/efficient_mlm_m0.40. Unfortunately this checkpoint depends on code that isn't part of the official transformers
library. Additionally, the checkpoints contains unused weights due to a bug.
This checkpoint fixes the unused weights issue and uses the RobertaPreLayerNorm model from the transformers
library.
- Task
- Fill-Mask
- Library
- transformers
- Type
- roberta-prelayernorm
- Created
- Nov 15, 2022
- Updated
- Nov 15, 2022