Downloads · 30 days
15
4% of all-time downloads
msmarco-distilbert-word2vec256k-MLM_445k_emb_updated
vocab-transformers/msmarco-distilbert-word2vec256k-MLM_445k_emb_updated
msmarco-distilbert-word2vec256k-MLM_445k_emb_updated is a fill-mask model from vocab-transformers. Use it when you need the model to fill a missing word. It is set up for transformers.
This model is based on nicoladecao/msmarco-word2vec256000-distilbert-base-uncased with a 256k sized vocabulary initialized with word2vec.
All-time downloads
405
Public
Repo size
3.8 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt1.9 GB · 67%
At a glance
- Task
- Fill-Mask
- Library
- transformers
- Model type
- distilbert
- Access
- Public
- Created
- Mar 2, 2022
- Updated
- Feb 21, 2022
- SHA
- 7b8f2c08
Try a prompt
Model card
From the Hugging Face model README
Model
This model is based on nicoladecao/msmarco-word2vec256000-distilbert-base-uncased with a 256k sized vocabulary initialized with word2vec.
This model has been trained with MLM on the MS MARCO corpus collection for 445k steps. See train_mlm.py for the train script. It was run on 2x V100 GPUs.
Note: Token embeddings where updated!
- Task
- Fill-Mask
- Library
- transformers
- Type
- distilbert
- Created
- Mar 2, 2022
- Updated
- Feb 21, 2022