Downloads · 30 days
24
3% of all-time downloads
dense_encoder-distilbert-frozen_emb
vocab-transformers/dense_encoder-distilbert-frozen_emb
dense_encoder-distilbert-frozen_emb is a feature extraction model from vocab-transformers. Use it when you need embeddings to search or compare text. It is set up for transformers.
This model is a distilbert-base-uncased model trained for 30 epochs (235k steps), 64 batch size with MarginMSE Loss on MS MARCO dataset.
All-time downloads
788
Public
Repo size
531 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin265 MB · 100%
At a glance
- Task
- Feature Extraction
- Library
- transformers
- Model type
- distilbert
- Access
- Public
- Created
- Apr 5, 2022
- Updated
- Apr 5, 2022
- SHA
- 8cb4a54e
Model card
From the Hugging Face model README
Dense Encoder - Distilbert - Frozen Token Embeddings
This model is a distilbert-base-uncased model trained for 30 epochs (235k steps), 64 batch size with MarginMSE Loss on MS MARCO dataset.
The token embeddings were frozen.
| Dataset | Model with updated token embeddings | Model with frozen embeddings |
|---|---|---|
| TREC-DL 19 | 70.68 | 68.60 |
| TREC-DL 20 | 67.69 | 70.21 |
| FiQA | 28.89 | 28.60 |
| Robust04 | 39.56 | 39.08 |
| TREC-COVID v2 | 69.80 | 69.84 |
| TREC-NEWS | 37.97 | 38.27 |
| Avg. 4 BEIR tasks | 44.06 | 43.95 |
- Task
- Feature Extraction
- Library
- transformers
- Type
- distilbert
- Created
- Apr 5, 2022
- Updated
- Apr 5, 2022