Downloads · 30 days
12
0% of all-time downloads
msmarco-distilbert-margin-mse
GPL/msmarco-distilbert-margin-mse
msmarco-distilbert-margin-mse is a feature extraction model from GPL. Use it when you need embeddings to search or compare text. It is set up for transformers.
This is the zero-shot baseline model in the paper "GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrieval"
All-time downloads
13.9K
Public
Repo size
531 MB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.bin265 MB · 100%
At a glance
- Task
- Feature Extraction
- Library
- transformers
- Model type
- distilbert
- Access
- Public
- Created
- Mar 2, 2022
- Updated
- Dec 15, 2021
- SHA
- 3fbae3e9
Model card
From the Hugging Face model README
This is the zero-shot baseline model in the paper "GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrieval"
The training setup:
- Start from
distilbert-base-uncased; - Mine 50 hard negatives for each query on MS MARCO with
sentence-transformers/msmarco-distilbert-base-v3andsentence-transformers/msmarco-MiniLM-L-6-v3; - Do Margin-MSE training on the tuples (including queries, gold relevant, and hard negatives) with the teacher model
cross-encoder/ms-marco-MiniLM-L-6-v2for 70K steps with batch size 75, max. sequence-length 350.
- Task
- Feature Extraction
- Library
- transformers
- Type
- distilbert
- Created
- Mar 2, 2022
- Updated
- Dec 15, 2021