Downloads · 30 days
33
7% of all-time downloads
msmarco-distilbert-word2vec256k-MLM_230k
vocab-transformers/msmarco-distilbert-word2vec256k-MLM_230k
msmarco-distilbert-word2vec256k-MLM_230k is a fill-mask model from vocab-transformers. Use it when you need the model to fill a missing word. It is set up for transformers.
This model is based on nicoladecao/msmarco-word2vec256000-distilbert-base-uncased with a 256k sized vocabulary initialized with word2vec.
All-time downloads
477
Public
Repo size
2.3 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin962 MB · 73%
At a glance
- Task
- Fill-Mask
- Library
- transformers
- Model type
- distilbert
- Access
- Public
- Created
- Mar 2, 2022
- Updated
- Feb 22, 2022
- SHA
- 77b49f5e
Try a prompt
Model card
From the Hugging Face model README
Model
This model is based on nicoladecao/msmarco-word2vec256000-distilbert-base-uncased with a 256k sized vocabulary initialized with word2vec.
This model has been trained with MLM on the MS MARCO corpus collection for 230k steps. See train_mlm.py for the train script. It was run on 2x V100 GPUs. The word embedding matrix was frozen.
- Task
- Fill-Mask
- Library
- transformers
- Type
- distilbert
- Created
- Mar 2, 2022
- Updated
- Feb 22, 2022