Downloads · 30 days
16
0% of all-time downloads
ColBERTv1.0-german-mmarcoDE
AdrienB134/ColBERTv1.0-german-mmarcoDE
ColBERTv1.0-german-mmarcoDE is a machine learning model from AdrienB134. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as mit.
The model is initialized from the dbmdz/bert-base-german-uncased checkpoint and fine-tuned on 10M triples via pairwise softmax cross-entropy loss over the computed scores of the positive and negative passages associat…
All-time downloads
21.5K
Public
Parameters
110M
440 MB on disk
Likes
3
Public
Repo makeup
Click a slice to open those files.
.safetensors440 MB · 100%
At a glance
- Library
- transformers
- License
- mit
- Model type
- bert
- Access
- Public
- Created
- Feb 29, 2024
- Updated
- Aug 24, 2024
- SHA
- d974d689
Datasets
Model card
From the Hugging Face model README
Training
Details
The model is initialized from the dbmdz/bert-base-german-uncased checkpoint and fine-tuned on 10M triples via pairwise softmax cross-entropy loss over the computed scores of the positive and negative passages associated to a query. It was trained on a single Tesla A100 GPU with 40GBs of memory during 200k steps with 10% of warmup steps using a batch size of 96 and the AdamW optimizer with a constant learning rate of 3e-06. Total training time was around 12 hours.
Data
The model is fine-tuned on the German version of the mMARCO dataset, a multi-lingual machine-translated version of the MS MARCO dataset. The triples are sampled from the ~39.8M triples of triples.train.small.tsv
Evaluation
The model is evaluated on the smaller development set of mMARCO-de, which consists of 6,980 queries for a corpus of 8.8M candidate passages. We report the mean reciprocal rank (MRR) and recall at various cut-offs (R@k).
| model | Vocab. | #Param. | Size | MRR@10 | R@50 | R@1000 |
|---|---|---|---|---|---|---|
| ColBERTv1.0-german-mmarcoDE | german | 110M | 440MB | 26.62 | 63.66 | 68.32 |
- Library
- transformers
- Type
- bert
- License
- mit
- Languages
- de
- Created
- Feb 29, 2024
- Updated
- Aug 24, 2024