Downloads · 30 days
1.6K
7% of all-time downloads
Alibaba-NLP_-_gte-Qwen2-1.5B-instruct-gguf
RichardErkhov/Alibaba-NLP_-_gte-Qwen2-1.5B-instruct-gguf
Alibaba-NLP_-_gte-Qwen2-1.5B-instruct-gguf is a machine learning model from RichardErkhov. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
All-time downloads
22.6K
Public
Repo size
21.1 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf24.4 GB · 100%
At a glance
- Access
- Public
- Created
- Jul 14, 2024
- Updated
- Jul 14, 2024
- SHA
- 8833356f
Model card
From the Hugging Face model README
Quantization made by Richard Erkhov.
gte-Qwen2-1.5B-instruct - GGUF
- Model creator: https://huggingface.co/Alibaba-NLP/
- Original model: https://huggingface.co/Alibaba-NLP/gte-Qwen2-1.5B-instruct/
Original model description:
tags:
- mteb
- sentence-transformers
- transformers
- Qwen2
- sentence-similarity license: apache-2.0 model-index:
- name: gte-qwen2-7B-instruct
results:
- dataset:
config: en
name: MTEB AmazonCounterfactualClassification (en)
revision: e8379541af4e31359cca9fbcf4b00f2671dba205
split: test
type: mteb/amazon_counterfactual
metrics:
- type: accuracy value: 83.98507462686567
- type: ap value: 50.93015252587014
- type: f1 value: 78.50416599051215 task: type: Classification
- dataset:
config: default
name: MTEB AmazonPolarityClassification
revision: e2d317d38cd51312af73b3d32a06d1a08b442046
split: test
type: mteb/amazon_polarity
metrics:
- type: accuracy value: 96.61065
- type: ap value: 94.89174052954196
- type: f1 value: 96.60942596940565 task: type: Classification
- dataset:
config: en
name: MTEB AmazonReviewsClassification (en)
revision: 1399c76144fd37290681b995c656ef9b2e06e26d
split: test
type: mteb/amazon_reviews_multi
metrics:
- type: accuracy value: 55.614000000000004
- type: f1 value: 54.90553480294904 task: type: Classification
- dataset:
config: default
name: MTEB ArguAna
revision: c22ab2a51041ffd869aaddef7af8d8215647e41a
split: test
type: mteb/arguana
metrics:
- type: map_at_1 value: 45.164
- type: map_at_10 value: 61.519
- type: map_at_100 value: 61.769
- type: map_at_1000 value: 61.769
- type: map_at_3 value: 57.443999999999996
- type: map_at_5 value: 60.058
- type: mrr_at_1 value: 46.088
- type: mrr_at_10 value: 61.861
- type: mrr_at_100 value: 62.117999999999995
- type: mrr_at_1000 value: 62.117999999999995
- type: mrr_at_3 value: 57.729
- type: mrr_at_5 value: 60.392
- type: ndcg_at_1 value: 45.164
- type: ndcg_at_10 value: 69.72
- type: ndcg_at_100 value: 70.719
- type: ndcg_at_1000 value: 70.719
- type: ndcg_at_3 value: 61.517999999999994
- type: ndcg_at_5 value: 66.247
- type: precision_at_1 value: 45.164
- type: precision_at_10 value: 9.545
- type: precision_at_100 value: 0.996
- type: precision_at_1000 value: 0.1
- type: precision_at_3 value: 24.443
- type: precision_at_5 value: 16.97
- type: recall_at_1 value: 45.164
- type: recall_at_10 value: 95.448
- type: recall_at_100 value: 99.644
- type: recall_at_1000 value: 99.644
- type: recall_at_3 value: 73.329
- type: recall_at_5 value: 84.851 task: type: Retrieval
- dataset:
config: default
name: MTEB ArxivClusteringP2P
revision: a122ad7f3f0291bf49cc6f4d32aa80929df69d5d
split: test
type: mteb/arxiv-clustering-p2p
metrics:
- type: v_measure value: 50.511868162026175 task: type: Clustering
- dataset:
config: default
name: MTEB ArxivClusteringS2S
revision: f910caf1a6075f7329cdf8c1a6135696f37dbd53
split: test
type: mteb/arxiv-clustering-s2s
metrics:
- type: v_measure value: 45.007803189284004 task: type: Clustering
- dataset:
config: default
name: MTEB AskUbuntuDupQuestions
revision: 2000358ca161889fa9c082cb41daa8dcfb161a54
split: test
type: mteb/askubuntudupquestions-reranking
metrics:
- type: map value: 64.55292107723382
- type: mrr value: 77.66158818097877 task: type: Reranking
- dataset:
config: default
name: MTEB BIOSSES
revision: d3fb88f8f02e40887cd149695127462bbcf29b4a
split: test
type: mteb/biosses-sts
metrics:
- type: cos_sim_pearson value: 85.65459047085452
- type: cos_sim_spearman value: 82.10729255710761
- type: euclidean_pearson value: 82.78079159312476
- type: euclidean_spearman value: 80.50002701880933
- type: manhattan_pearson value: 82.41372641383016
- type: manhattan_spearman value: 80.57412509272639 task: type: STS
- dataset:
config: default
name: MTEB Banking77Classification
revision: 0fd18e25b25c072e09e0d92ab615fda904d66300
split: test
type: mteb/banking77
metrics:
- type: accuracy value: 87.30844155844156
- type: f1 value: 87.25307322443255 task: type: Classification
- dataset:
config: default
name: MTEB BiorxivClusteringP2P
revision: 65b79d1d13f80053f67aca9498d9402c2d9f1f40
split: test
type: mteb/biorxiv-clustering-p2p
metrics:
- type: v_measure value: 43.20754608934859 task: type: Clustering
- dataset:
config: default
name: MTEB BiorxivClusteringS2S
revision: 258694dd0231531bc1fd9de6ceb52a0853c6d908
split: test
type: mteb/biorxiv-clustering-s2s
metrics:
- type: v_measure value: 38.818037697335505 task: type: Clustering
- dataset:
config: default
name: MTEB CQADupstackAndroidRetrieval
revision: f46a197baaae43b4f621051089b82a364682dfeb
split: test
type: BeIR/cqadupstack
metrics:
- type: map_at_1 value: 35.423
- type: map_at_10 value: 47.198
- type: map_at_100 value: 48.899
- type: map_at_1000 value: 49.004
- type: map_at_3 value: 43.114999999999995
- type: map_at_5 value: 45.491
- type: mrr_at_1 value: 42.918
- type: mrr_at_10 value: 53.299
- type: mrr_at_100 value: 54.032000000000004
- type: mrr_at_1000 value: 54.055
- type: mrr_at_3 value: 50.453
- type: mrr_at_5 value: 52.205999999999996
- type: ndcg_at_1 value: 42.918
- type: ndcg_at_10 value: 53.98
- type: ndcg_at_100 value: 59.57
- type: ndcg_at_1000 value: 60.879000000000005
- type: ndcg_at_3 value: 48.224000000000004
- type: ndcg_at_5 value: 50.998
- type: precision_at_1 value: 42.918
- type: precision_at_10 value: 10.299999999999999
- type: precision_at_100 value: 1.687
- type: precision_at_1000 value: 0.211
- type: precision_at_3 value: 22.842000000000002
- type: precision_at_5 value: 16.681
- type: recall_at_1 value: 35.423
- type: recall_at_10 value: 66.824
- type: recall_at_100 value: 89.564
- type: recall_at_1000 value: 97.501
- type: recall_at_3 value: 50.365
- type: recall_at_5 value: 57.921 task: type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackEnglishRetrieval
revision: ad9991cb51e31e31e430383c75ffb2885547b5f0
split: test
type: BeIR/cqadupstack
metrics:
- type: map_at_1 value: 33.205
- type: map_at_10 value: 44.859
- type: map_at_100 value: 46.135
- type: map_at_1000 value: 46.259
- type: map_at_3 value: 41.839
- type: map_at_5 value: 43.662
- type: mrr_at_1 value: 41.146
- type: mrr_at_10 value: 50.621
- type: mrr_at_100 value: 51.207
- type: mrr_at_1000 value: 51.246
- type: mrr_at_3 value: 48.535000000000004
- type: mrr_at_5 value: 49.818
- type: ndcg_at_1 value: 41.146
- type: ndcg_at_10 value: 50.683
- type: ndcg_at_100 value: 54.82
- type: ndcg_at_1000 value: 56.69
- type: ndcg_at_3 value: 46.611000000000004
- type: ndcg_at_5 value: 48.66
- type: precision_at_1 value: 41.146
- type: precision_at_10 value: 9.439
- type: precision_at_100 value: 1.465
- type: precision_at_1000 value: 0.194
- type: precision_at_3 value: 22.59
- type: precision_at_5 value: 15.86
- type: recall_at_1 value: 33.205
- type: recall_at_10 value: 61.028999999999996
- type: recall_at_100 value: 78.152
- type: recall_at_1000 value: 89.59700000000001
- type: recall_at_3 value: 49.05
- type: recall_at_5 value: 54.836 task: type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackGamingRetrieval
revision: 4885aa143210c98657558c04aaf3dc47cfb54340
split: test
type: BeIR/cqadupstack
metrics:
- type: map_at_1 value: 41.637
- type: map_at_10 value: 55.162
- type: map_at_100 value: 56.142
- type: map_at_1000 value: 56.188
- type: map_at_3 value: 51.564
- type: map_at_5 value: 53.696
- type: mrr_at_1 value: 47.524
- type: mrr_at_10 value: 58.243
- type: mrr_at_100 value: 58.879999999999995
- type: mrr_at_1000 value: 58.9
- type: mrr_at_3 value: 55.69499999999999
- type: mrr_at_5 value: 57.284
- type: ndcg_at_1 value: 47.524
- type: ndcg_at_10 value: 61.305
- type: ndcg_at_100 value: 65.077
- type: ndcg_at_1000 value: 65.941
- type: ndcg_at_3 value: 55.422000000000004
- type: ndcg_at_5 value: 58.516
- type: precision_at_1 value: 47.524
- type: precision_at_10 value: 9.918000000000001
- type: precision_at_100 value: 1.276
- type: precision_at_1000 value: 0.13899999999999998
- type: precision_at_3 value: 24.765
- type: precision_at_5 value: 17.204
- type: recall_at_1 value: 41.637
- type: recall_at_10 value: 76.185
- type: recall_at_100 value: 92.149
- type: recall_at_1000 value: 98.199
- type: recall_at_3 value: 60.856
- type: recall_at_5 value: 68.25099999999999 task: type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackGisRetrieval
revision: 5003b3064772da1887988e05400cf3806fe491f2
split: test
type: BeIR/cqadupstack
metrics:
- type: map_at_1 value: 26.27
- type: map_at_10 value: 37.463
- type: map_at_100 value: 38.434000000000005
- type: map_at_1000 value: 38.509
- type: map_at_3 value: 34.226
- type: map_at_5 value: 36.161
- type: mrr_at_1 value: 28.588
- type: mrr_at_10 value: 39.383
- type: mrr_at_100 value: 40.23
- type: mrr_at_1000 value: 40.281
- type: mrr_at_3 value: 36.422
- type: mrr_at_5 value: 38.252
- type: ndcg_at_1 value: 28.588
- type: ndcg_at_10 value: 43.511
- type: ndcg_at_100 value: 48.274
- type: ndcg_at_1000 value: 49.975
- type: ndcg_at_3 value: 37.319
- type: ndcg_at_5 value: 40.568
- type: precision_at_1 value: 28.588
- type: precision_at_10 value: 6.893000000000001
- type: precision_at_100 value: 0.9900000000000001
- type: precision_at_1000 value: 0.117
- type: precision_at_3 value: 16.347
- type: precision_at_5 value: 11.661000000000001
- type: recall_at_1 value: 26.27
- type: recall_at_10 value: 60.284000000000006
- type: recall_at_100 value: 81.902
- type: recall_at_1000 value: 94.43
- type: recall_at_3 value: 43.537
- type: recall_at_5 value: 51.475 task: type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackMathematicaRetrieval
revision: 90fceea13679c63fe563ded68f3b6f06e50061de
split: test
type: BeIR/cqadupstack
metrics:
- type: map_at_1 value: 18.168
- type: map_at_10 value: 28.410000000000004
- type: map_at_100 value: 29.78
- type: map_at_1000 value: 29.892999999999997
- type: map_at_3 value: 25.238
- type: map_at_5 value: 26.96
- type: mrr_at_1 value: 23.507
- type: mrr_at_10 value: 33.382
- type: mrr_at_100 value: 34.404
- type: mrr_at_1000 value: 34.467999999999996
- type: mrr_at_3 value: 30.637999999999998
- type: mrr_at_5 value: 32.199
- type: ndcg_at_1 value: 23.507
- type: ndcg_at_10 value: 34.571000000000005
- type: ndcg_at_100 value: 40.663
- type: ndcg_at_1000 value: 43.236000000000004
- type: ndcg_at_3 value: 29.053
- type: ndcg_at_5 value: 31.563999999999997
- type: precision_at_1 value: 23.507
- type: precision_at_10 value: 6.654
- type: precision_at_100 value: 1.113
- type: precision_at_1000 value: 0.146
- type: precision_at_3 value: 14.427999999999999
- type: precision_at_5 value: 10.498000000000001
- type: recall_at_1 value: 18.168
- type: recall_at_10 value: 48.443000000000005
- type: recall_at_100 value: 74.47
- type: recall_at_1000 value: 92.494
- type: recall_at_3 value: 33.379999999999995
- type: recall_at_5 value: 39.76 task: type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackPhysicsRetrieval
revision: 79531abbd1fb92d06c6d6315a0cbbbf5bb247ea4
split: test
type: BeIR/cqadupstack
metrics:
- type: map_at_1 value: 32.39
- type: map_at_10 value: 44.479
- type: map_at_100 value: 45.977000000000004
- type: map_at_1000 value: 46.087
- type: map_at_3 value: 40.976
- type: map_at_5 value: 43.038
- type: mrr_at_1 value: 40.135
- type: mrr_at_10 value: 50.160000000000004
- type: mrr_at_100 value: 51.052
- type: mrr_at_1000 value: 51.087
- type: mrr_at_3 value: 47.818
- type: mrr_at_5 value: 49.171
- type: ndcg_at_1 value: 40.135
- type: ndcg_at_10 value: 50.731
- type: ndcg_at_100 value: 56.452000000000005
- type: ndcg_at_1000 value: 58.123000000000005
- type: ndcg_at_3 value: 45.507
- type: ndcg_at_5 value: 48.11
- type: precision_at_1 value: 40.135
- type: precision_at_10 value: 9.192
- type: precision_at_100 value: 1.397
- type: precision_at_1000 value: 0.169
- type: precision_at_3 value: 21.816
- type: precision_at_5 value: 15.476
- type: recall_at_1 value: 32.39
- type: recall_at_10 value: 63.597
- type: recall_at_100 value: 86.737
- type: recall_at_1000 value: 97.039
- type: recall_at_3 value: 48.906
- type: recall_at_5 value: 55.659000000000006 task: type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackProgrammersRetrieval
revision: 6184bc1440d2dbc7612be22b50686b8826d22b32
split: test
type: BeIR/cqadupstack
metrics:
- type: map_at_1 value: 28.397
- type: map_at_10 value: 39.871
- type: map_at_100 value: 41.309000000000005
- type: map_at_1000 value: 41.409
- type: map_at_3 value: 36.047000000000004
- type: map_at_5 value: 38.104
- type: mrr_at_1 value: 34.703
- type: mrr_at_10 value: 44.773
- type: mrr_at_100 value: 45.64
- type: mrr_at_1000 value: 45.678999999999995
- type: mrr_at_3 value: 41.705
- type: mrr_at_5 value: 43.406
- type: ndcg_at_1 value: 34.703
- type: ndcg_at_10 value: 46.271
- type: ndcg_at_100 value: 52.037
- type: ndcg_at_1000 value: 53.81700000000001
- type: ndcg_at_3 value: 39.966
- type: ndcg_at_5 value: 42.801
- type: precision_at_1 value: 34.703
- type: precision_at_10 value: 8.744
- type: precision_at_100 value: 1.348
- type: precision_at_1000 value: 0.167
- type: precision_at_3 value: 19.102
- type: precision_at_5 value: 13.836
- type: recall_at_1 value: 28.397
- type: recall_at_10 value: 60.299
- type: recall_at_100 value: 84.595
- type: recall_at_1000 value: 96.155
- type: recall_at_3 value: 43.065
- type: recall_at_5 value: 50.371 task: type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackRetrieval
revision: 4ffe81d471b1924886b33c7567bfb200e9eec5c4
split: test
type: BeIR/cqadupstack
metrics:
- type: map_at_1 value: 28.044333333333338
- type: map_at_10 value: 38.78691666666666
- type: map_at_100 value: 40.113
- type: map_at_1000 value: 40.22125
- type: map_at_3 value: 35.52966666666667
- type: map_at_5 value: 37.372749999999996
- type: mrr_at_1 value: 33.159083333333335
- type: mrr_at_10 value: 42.913583333333335
- type: mrr_at_100 value: 43.7845
- type: mrr_at_1000 value: 43.830333333333336
- type: mrr_at_3 value: 40.29816666666667
- type: mrr_at_5 value: 41.81366666666667
- type: ndcg_at_1 value: 33.159083333333335
- type: ndcg_at_10 value: 44.75750000000001
- type: ndcg_at_100 value: 50.13658333333334
- type: ndcg_at_1000 value: 52.037
- type: ndcg_at_3 value: 39.34258333333334
- type: ndcg_at_5 value: 41.93708333333333
- type: precision_at_1 value: 33.159083333333335
- type: precision_at_10 value: 7.952416666666667
- type: precision_at_100 value: 1.2571666666666668
- type: precision_at_1000 value: 0.16099999999999998
- type: precision_at_3 value: 18.303833333333337
- type: precision_at_5 value: 13.057083333333333
- type: recall_at_1 value: 28.044333333333338
- type: recall_at_10 value: 58.237249999999996
- type: recall_at_100 value: 81.35391666666666
- type: recall_at_1000 value: 94.21283333333334
- type: recall_at_3 value: 43.32341666666667
- type: recall_at_5 value: 49.94908333333333 task: type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackStatsRetrieval
revision: 65ac3a16b8e91f9cee4c9828cc7c335575432a2a
split: test
type: BeIR/cqadupstack
metrics:
- type: map_at_1 value: 27.838
- type: map_at_10 value: 36.04
- type: map_at_100 value: 37.113
- type: map_at_1000 value: 37.204
- type: map_at_3 value: 33.585
- type: map_at_5 value: 34.845
- type: mrr_at_1 value: 30.982
- type: mrr_at_10 value: 39.105000000000004
- type: mrr_at_100 value: 39.98
- type: mrr_at_1000 value: 40.042
- type: mrr_at_3 value: 36.912
- type: mrr_at_5 value: 38.062000000000005
- type: ndcg_at_1 value: 30.982
- type: ndcg_at_10 value: 40.982
- type: ndcg_at_100 value: 46.092
- type: ndcg_at_1000 value: 48.25
- type: ndcg_at_3 value: 36.41
- type: ndcg_at_5 value: 38.379999999999995
- type: precision_at_1 value: 30.982
- type: precision_at_10 value: 6.534
- type: precision_at_100 value: 0.9820000000000001
- type: precision_at_1000 value: 0.124
- type: precision_at_3 value: 15.745999999999999
- type: precision_at_5 value: 10.828
- type: recall_at_1 value: 27.838
- type: recall_at_10 value: 52.971000000000004
- type: recall_at_100 value: 76.357
- type: recall_at_1000 value: 91.973
- type: recall_at_3 value: 40.157
- type: recall_at_5 value: 45.147999999999996 task: type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackTexRetrieval
revision: 46989137a86843e03a6195de44b09deda022eec7
split: test
type: BeIR/cqadupstack
metrics:
- type: map_at_1 value: 19.059
- type: map_at_10 value: 27.454
- type: map_at_100 value: 28.736
- type: map_at_1000 value: 28.865000000000002
- type: map_at_3 value: 24.773999999999997
- type: map_at_5 value: 26.266000000000002
- type: mrr_at_1 value: 23.125
- type: mrr_at_10 value: 31.267
- type: mrr_at_100 value: 32.32
- type: mrr_at_1000 value: 32.394
- type: mrr_at_3 value: 28.894
- type: mrr_at_5 value: 30.281000000000002
- type: ndcg_at_1 value: 23.125
- type: ndcg_at_10 value: 32.588
- type: ndcg_at_100 value: 38.432
- type: ndcg_at_1000 value: 41.214
- type: ndcg_at_3 value: 27.938000000000002
- type: ndcg_at_5 value: 30.127
- type: precision_at_1 value: 23.125
- type: precision_at_10 value: 5.9639999999999995
- type: precision_at_100 value: 1.047
- type: precision_at_1000 value: 0.148
- type: precision_at_3 value: 13.294
- type: precision_at_5 value: 9.628
- type: recall_at_1 value: 19.059
- type: recall_at_10 value: 44.25
- type: recall_at_100 value: 69.948
- type: recall_at_1000 value: 89.35300000000001
- type: recall_at_3 value: 31.114000000000004
- type: recall_at_5 value: 36.846000000000004 task: type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackUnixRetrieval
revision: 6c6430d3a6d36f8d2a829195bc5dc94d7e063e53
split: test
type: BeIR/cqadupstack
metrics:
- type: map_at_1 value: 28.355999999999998
- type: map_at_10 value: 39.055
- type: map_at_100 value: 40.486
- type: map_at_1000 value: 40.571
- type: map_at_3 value: 35.69
- type: map_at_5 value: 37.605
- type: mrr_at_1 value: 33.302
- type: mrr_at_10 value: 42.986000000000004
- type: mrr_at_100 value: 43.957
- type: mrr_at_1000 value: 43.996
- type: mrr_at_3 value: 40.111999999999995
- type: mrr_at_5 value: 41.735
- type: ndcg_at_1 value: 33.302
- type: ndcg_at_10 value: 44.962999999999994
- type: ndcg_at_100 value: 50.917
- type: ndcg_at_1000 value: 52.622
- type: ndcg_at_3 value: 39.182
- type: ndcg_at_5 value: 41.939
- type: precision_at_1 value: 33.302
- type: precision_at_10 value: 7.779999999999999
- type: precision_at_100 value: 1.203
- type: precision_at_1000 value: 0.145
- type: precision_at_3 value: 18.035
- type: precision_at_5 value: 12.873000000000001
- type: recall_at_1 value: 28.355999999999998
- type: recall_at_10 value: 58.782000000000004
- type: recall_at_100 value: 84.02199999999999
- type: recall_at_1000 value: 95.511
- type: recall_at_3 value: 43.126999999999995
- type: recall_at_5 value: 50.14999999999999 task: type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackWebmastersRetrieval
revision: 160c094312a0e1facb97e55eeddb698c0abe3571
split: test
type: BeIR/cqadupstack
metrics:
- type: map_at_1 value: 27.391
- type: map_at_10 value: 37.523
- type: map_at_100 value: 39.312000000000005
- type: map_at_1000 value: 39.54
- type: map_at_3 value: 34.231
- type: map_at_5 value: 36.062
- type: mrr_at_1 value: 32.016
- type: mrr_at_10 value: 41.747
- type: mrr_at_100 value: 42.812
- type: mrr_at_1000 value: 42.844
- type: mrr_at_3 value: 39.129999999999995
- type: mrr_at_5 value: 40.524
- type: ndcg_at_1 value: 32.016
- type: ndcg_at_10 value: 43.826
- type: ndcg_at_100 value: 50.373999999999995
- type: ndcg_at_1000 value: 52.318
- type: ndcg_at_3 value: 38.479
- type: ndcg_at_5 value: 40.944
- type: precision_at_1 value: 32.016
- type: precision_at_10 value: 8.280999999999999
- type: precision_at_100 value: 1.6760000000000002
- type: precision_at_1000 value: 0.25
- type: precision_at_3 value: 18.05
- type: precision_at_5 value: 13.083
- type: recall_at_1 value: 27.391
- type: recall_at_10 value: 56.928999999999995
- type: recall_at_100 value: 85.169
- type: recall_at_1000 value: 96.665
- type: recall_at_3 value: 42.264
- type: recall_at_5 value: 48.556 task: type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackWordpressRetrieval
revision: 4ffe81d471b1924886b33c7567bfb200e9eec5c4
split: test
type: BeIR/cqadupstack
metrics:
- type: map_at_1 value: 18.398
- type: map_at_10 value: 27.929
- type: map_at_100 value: 29.032999999999998
- type: map_at_1000 value: 29.126
- type: map_at_3 value: 25.070999999999998
- type: map_at_5 value: 26.583000000000002
- type: mrr_at_1 value: 19.963
- type: mrr_at_10 value: 29.997
- type: mrr_at_100 value: 30.9
- type: mrr_at_1000 value: 30.972
- type: mrr_at_3 value: 27.264
- type: mrr_at_5 value: 28.826
- type: ndcg_at_1 value: 19.963
- type: ndcg_at_10 value: 33.678999999999995
- type: ndcg_at_100 value: 38.931
- type: ndcg_at_1000 value: 41.379
- type: ndcg_at_3 value: 28.000000000000004
- type: ndcg_at_5 value: 30.637999999999998
- type: precision_at_1 value: 19.963
- type: precision_at_10 value: 5.7299999999999995
- type: precision_at_100 value: 0.902
- type: precision_at_1000 value: 0.122
- type: precision_at_3 value: 12.631
- type: precision_at_5 value: 9.057
- type: recall_at_1 value: 18.398
- type: recall_at_10 value: 49.254
- type: recall_at_100 value: 73.182
- type: recall_at_1000 value: 91.637
- type: recall_at_3 value: 34.06
- type: recall_at_5 value: 40.416000000000004 task: type: Retrieval
- dataset:
config: default
name: MTEB ClimateFEVER
revision: 47f2ac6acb640fc46020b02a5b59fdda04d39380
split: test
type: mteb/climate-fever
metrics:
- type: map_at_1 value: 19.681
- type: map_at_10 value: 32.741
- type: map_at_100 value: 34.811
- type: map_at_1000 value: 35.003
- type: map_at_3 value: 27.697
- type: map_at_5 value: 30.372
- type: mrr_at_1 value: 44.951
- type: mrr_at_10 value: 56.34400000000001
- type: mrr_at_100 value: 56.961
- type: mrr_at_1000 value: 56.987
- type: mrr_at_3 value: 53.681
- type: mrr_at_5 value: 55.407
- type: ndcg_at_1 value: 44.951
- type: ndcg_at_10 value: 42.905
- type: ndcg_at_100 value: 49.95
- type: ndcg_at_1000 value: 52.917
- type: ndcg_at_3 value: 36.815
- type: ndcg_at_5 value: 38.817
- type: precision_at_1 value: 44.951
- type: precision_at_10 value: 12.989999999999998
- type: precision_at_100 value: 2.068
- type: precision_at_1000 value: 0.263
- type: precision_at_3 value: 27.275
- type: precision_at_5 value: 20.365
- type: recall_at_1 value: 19.681
- type: recall_at_10 value: 48.272999999999996
- type: recall_at_100 value: 71.87400000000001
- type: recall_at_1000 value: 87.929
- type: recall_at_3 value: 32.653999999999996
- type: recall_at_5 value: 39.364 task: type: Retrieval
- dataset:
config: default
name: MTEB DBPedia
revision: c0f706b76e590d620bd6618b3ca8efdd34e2d659
split: test
type: mteb/dbpedia
metrics:
- type: map_at_1 value: 10.231
- type: map_at_10 value: 22.338
- type: map_at_100 value: 31.927
- type: map_at_1000 value: 33.87
- type: map_at_3 value: 15.559999999999999
- type: map_at_5 value: 18.239
- type: mrr_at_1 value: 75.0
- type: mrr_at_10 value: 81.303
- type: mrr_at_100 value: 81.523
- type: mrr_at_1000 value: 81.53
- type: mrr_at_3 value: 80.083
- type: mrr_at_5 value: 80.758
- type: ndcg_at_1 value: 64.625
- type: ndcg_at_10 value: 48.687000000000005
- type: ndcg_at_100 value: 52.791
- type: ndcg_at_1000 value: 60.041999999999994
- type: ndcg_at_3 value: 53.757999999999996
- type: ndcg_at_5 value: 50.76500000000001
- type: precision_at_1 value: 75.0
- type: precision_at_10 value: 38.3
- type: precision_at_100 value: 12.025
- type: precision_at_1000 value: 2.3970000000000002
- type: precision_at_3 value: 55.417
- type: precision_at_5 value: 47.5
- type: recall_at_1 value: 10.231
- type: recall_at_10 value: 27.697
- type: recall_at_100 value: 57.409
- type: recall_at_1000 value: 80.547
- type: recall_at_3 value: 16.668
- type: recall_at_5 value: 20.552 task: type: Retrieval
- dataset:
config: default
name: MTEB EmotionClassification
revision: 4f58c6b202a23cf9a4da393831edf4f9183cad37
split: test
type: mteb/emotion
metrics:
- type: accuracy value: 61.365
- type: f1 value: 56.7540827912991 task: type: Classification
- dataset:
config: default
name: MTEB FEVER
revision: bea83ef9e8fb933d90a2f1d5515737465d613e12
split: test
type: mteb/fever
metrics:
- type: map_at_1 value: 83.479
- type: map_at_10 value: 88.898
- type: map_at_100 value: 89.11
- type: map_at_1000 value: 89.12400000000001
- type: map_at_3 value: 88.103
- type: map_at_5 value: 88.629
- type: mrr_at_1 value: 89.934
- type: mrr_at_10 value: 93.91000000000001
- type: mrr_at_100 value: 93.937
- type: mrr_at_1000 value: 93.938
- type: mrr_at_3 value: 93.62700000000001
- type: mrr_at_5 value: 93.84599999999999
- type: ndcg_at_1 value: 89.934
- type: ndcg_at_10 value: 91.574
- type: ndcg_at_100 value: 92.238
- type: ndcg_at_1000 value: 92.45
- type: ndcg_at_3 value: 90.586
- type: ndcg_at_5 value: 91.16300000000001
- type: precision_at_1 value: 89.934
- type: precision_at_10 value: 10.555
- type: precision_at_100 value: 1.1159999999999999
- type: precision_at_1000 value: 0.11499999999999999
- type: precision_at_3 value: 33.588
- type: precision_at_5 value: 20.642
- type: recall_at_1 value: 83.479
- type: recall_at_10 value: 94.971
- type: recall_at_100 value: 97.397
- type: recall_at_1000 value: 98.666
- type: recall_at_3 value: 92.24799999999999
- type: recall_at_5 value: 93.797 task: type: Retrieval
- dataset:
config: default
name: MTEB FiQA2018
revision: 27a168819829fe9bcd655c2df245fb19452e8e06
split: test
type: mteb/fiqa
metrics:
- type: map_at_1 value: 27.16
- type: map_at_10 value: 45.593
- type: map_at_100 value: 47.762
- type: map_at_1000 value: 47.899
- type: map_at_3 value: 39.237
- type: map_at_5 value: 42.970000000000006
- type: mrr_at_1 value: 52.623
- type: mrr_at_10 value: 62.637
- type: mrr_at_100 value: 63.169
- type: mrr_at_1000 value: 63.185
- type: mrr_at_3 value: 59.928000000000004
- type: mrr_at_5 value: 61.702999999999996
- type: ndcg_at_1 value: 52.623
- type: ndcg_at_10 value: 54.701
- type: ndcg_at_100 value: 61.263
- type: ndcg_at_1000 value: 63.134
- type: ndcg_at_3 value: 49.265
- type: ndcg_at_5 value: 51.665000000000006
- type: precision_at_1 value: 52.623
- type: precision_at_10 value: 15.185
- type: precision_at_100 value: 2.202
- type: precision_at_1000 value: 0.254
- type: precision_at_3 value: 32.767
- type: precision_at_5 value: 24.722
- type: recall_at_1 value: 27.16
- type: recall_at_10 value: 63.309000000000005
- type: recall_at_100 value: 86.722
- type: recall_at_1000 value: 97.505
- type: recall_at_3 value: 45.045
- type: recall_at_5 value: 54.02400000000001 task: type: Retrieval
- dataset:
config: default
name: MTEB HotpotQA
revision: ab518f4d6fcca38d87c25209f94beba119d02014
split: test
type: mteb/hotpotqa
metrics:
- type: map_at_1 value: 42.573
- type: map_at_10 value: 59.373
- type: map_at_100 value: 60.292
- type: map_at_1000 value: 60.358999999999995
- type: map_at_3 value: 56.159000000000006
- type: map_at_5 value: 58.123999999999995
- type: mrr_at_1 value: 85.14500000000001
- type: mrr_at_10 value: 89.25999999999999
- type: mrr_at_100 value: 89.373
- type: mrr_at_1000 value: 89.377
- type: mrr_at_3 value: 88.618
- type: mrr_at_5 value: 89.036
- type: ndcg_at_1 value: 85.14500000000001
- type: ndcg_at_10 value: 68.95
- type: ndcg_at_100 value: 71.95
- type: ndcg_at_1000 value: 73.232
- type: ndcg_at_3 value: 64.546
- type: ndcg_at_5 value: 66.945
- type: precision_at_1 value: 85.14500000000001
- type: precision_at_10 value: 13.865
- type: precision_at_100 value: 1.619
- type: precision_at_1000 value: 0.179
- type: precision_at_3 value: 39.703
- type: precision_at_5 value: 25.718000000000004
- type: recall_at_1 value: 42.573
- type: recall_at_10 value: 69.325
- type: recall_at_100 value: 80.932
- type: recall_at_1000 value: 89.446
- type: recall_at_3 value: 59.553999999999995
- type: recall_at_5 value: 64.294 task: type: Retrieval
- dataset:
config: default
name: MTEB ImdbClassification
revision: 3d86128a09e091d6018b6d26cad27f2739fc2db7
split: test
type: mteb/imdb
metrics:
- type: accuracy value: 95.8336
- type: ap value: 93.78862962194073
- type: f1 value: 95.83192650728371 task: type: Classification
- dataset:
config: default
name: MTEB MSMARCO
revision: c5a29a104738b98a9e76336939199e264163d4a0
split: dev
type: mteb/msmarco
metrics:
- type: map_at_1 value: 23.075000000000003
- type: map_at_10 value: 36.102000000000004
- type: map_at_100 value: 37.257
- type: map_at_1000 value: 37.3
- type: map_at_3 value: 32.144
- type: map_at_5 value: 34.359
- type: mrr_at_1 value: 23.711
- type: mrr_at_10 value: 36.671
- type: mrr_at_100 value: 37.763999999999996
- type: mrr_at_1000 value: 37.801
- type: mrr_at_3 value: 32.775
- type: mrr_at_5 value: 34.977000000000004
- type: ndcg_at_1 value: 23.711
- type: ndcg_at_10 value: 43.361
- type: ndcg_at_100 value: 48.839
- type: ndcg_at_1000 value: 49.88
- type: ndcg_at_3 value: 35.269
- type: ndcg_at_5 value: 39.224
- type: precision_at_1 value: 23.711
- type: precision_at_10 value: 6.866999999999999
- type: precision_at_100 value: 0.96
- type: precision_at_1000 value: 0.105
- type: precision_at_3 value: 15.096000000000002
- type: precision_at_5 value: 11.083
- type: recall_at_1 value: 23.075000000000003
- type: recall_at_10 value: 65.756
- type: recall_at_100 value: 90.88199999999999
- type: recall_at_1000 value: 98.739
- type: recall_at_3 value: 43.691
- type: recall_at_5 value: 53.15800000000001 task: type: Retrieval
- dataset:
config: en
name: MTEB MTOPDomainClassification (en)
revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
split: test
type: mteb/mtop_domain
metrics:
- type: accuracy value: 97.69493844049248
- type: f1 value: 97.55048089616261 task: type: Classification
- dataset:
config: en
name: MTEB MTOPIntentClassification (en)
revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
split: test
type: mteb/mtop_intent
metrics:
- type: accuracy value: 88.75968992248062
- type: f1 value: 72.26321223399123 task: type: Classification
- dataset:
config: en
name: MTEB MassiveIntentClassification (en)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy value: 82.40080699394754
- type: f1 value: 79.62590029057968 task: type: Classification
- dataset:
config: en
name: MTEB MassiveScenarioClassification (en)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy value: 84.49562878278414
- type: f1 value: 84.0040193313333 task: type: Classification
- dataset:
config: default
name: MTEB MedrxivClusteringP2P
revision: e7a26af6f3ae46b30dde8737f02c07b1505bcc73
split: test
type: mteb/medrxiv-clustering-p2p
metrics:
- type: v_measure value: 39.386760057101945 task: type: Clustering
- dataset:
config: default
name: MTEB MedrxivClusteringS2S
revision: 35191c8c0dca72d8ff3efcd72aa802307d469663
split: test
type: mteb/medrxiv-clustering-s2s
metrics:
- type: v_measure value: 37.89687154075537 task: type: Clustering
- dataset:
config: default
name: MTEB MindSmallReranking
revision: 3bdac13927fdc888b903db93b2ffdbd90b295a69
split: test
type: mteb/mind_small
metrics:
- type: map value: 33.94151656057482
- type: mrr value: 35.32684700746953 task: type: Reranking
- dataset:
config: default
name: MTEB NFCorpus
revision: ec0fa4fe99da2ff19ca1214b7966684033a58814
split: test
type: mteb/nfcorpus
metrics:
- type: map_at_1 value: 6.239999999999999
- type: map_at_10 value: 14.862
- type: map_at_100 value: 18.955
- type: map_at_1000 value: 20.694000000000003
- type: map_at_3 value: 10.683
- type: map_at_5 value: 12.674
- type: mrr_at_1 value: 50.15500000000001
- type: mrr_at_10 value: 59.697
- type: mrr_at_100 value: 60.095
- type: mrr_at_1000 value: 60.129999999999995
- type: mrr_at_3 value: 58.35900000000001
- type: mrr_at_5 value: 58.839
- type: ndcg_at_1 value: 48.452
- type: ndcg_at_10 value: 39.341
- type: ndcg_at_100 value: 35.866
- type: ndcg_at_1000 value: 45.111000000000004
- type: ndcg_at_3 value: 44.527
- type: ndcg_at_5 value: 42.946
- type: precision_at_1 value: 50.15500000000001
- type: precision_at_10 value: 29.536
- type: precision_at_100 value: 9.142
- type: precision_at_1000 value: 2.2849999999999997
- type: precision_at_3 value: 41.899
- type: precision_at_5 value: 37.647000000000006
- type: recall_at_1 value: 6.239999999999999
- type: recall_at_10 value: 19.278000000000002
- type: recall_at_100 value: 36.074
- type: recall_at_1000 value: 70.017
- type: recall_at_3 value: 12.066
- type: recall_at_5 value: 15.254000000000001 task: type: Retrieval
- dataset:
config: default
name: MTEB NQ
revision: b774495ed302d8c44a3a7ea25c90dbce03968f31
split: test
type: mteb/nq
metrics:
- type: map_at_1 value: 39.75
- type: map_at_10 value: 56.443
- type: map_at_100 value: 57.233999999999995
- type: map_at_1000 value: 57.249
- type: map_at_3 value: 52.032999999999994
- type: map_at_5 value: 54.937999999999995
- type: mrr_at_1 value: 44.728
- type: mrr_at_10 value: 58.939
- type: mrr_at_100 value: 59.489000000000004
- type: mrr_at_1000 value: 59.499
- type: mrr_at_3 value: 55.711999999999996
- type: mrr_at_5 value: 57.89
- type: ndcg_at_1 value: 44.728
- type: ndcg_at_10 value: 63.998999999999995
- type: ndcg_at_100 value: 67.077
- type: ndcg_at_1000 value: 67.40899999999999
- type: ndcg_at_3 value: 56.266000000000005
- type: ndcg_at_5 value: 60.88
- type: precision_at_1 value: 44.728
- type: precision_at_10 value: 10.09
- type: precision_at_100 value: 1.1809999999999998
- type: precision_at_1000 value: 0.121
- type: precision_at_3 value: 25.145
- type: precision_at_5 value: 17.822
- type: recall_at_1 value: 39.75
- type: recall_at_10 value: 84.234
- type: recall_at_100 value: 97.055
- type: recall_at_1000 value: 99.517
- type: recall_at_3 value: 64.851
- type: recall_at_5 value: 75.343 task: type: Retrieval
- dataset:
config: default
name: MTEB QuoraRetrieval
revision: None
split: test
type: mteb/quora
metrics:
- type: map_at_1 value: 72.085
- type: map_at_10 value: 86.107
- type: map_at_100 value: 86.727
- type: map_at_1000 value: 86.74
- type: map_at_3 value: 83.21
- type: map_at_5 value: 85.06
- type: mrr_at_1 value: 82.94
- type: mrr_at_10 value: 88.845
- type: mrr_at_100 value: 88.926
- type: mrr_at_1000 value: 88.927
- type: mrr_at_3 value: 87.993
- type: mrr_at_5 value: 88.62299999999999
- type: ndcg_at_1 value: 82.97
- type: ndcg_at_10 value: 89.645
- type: ndcg_at_100 value: 90.717
- type: ndcg_at_1000 value: 90.78
- type: ndcg_at_3 value: 86.99900000000001
- type: ndcg_at_5 value: 88.52600000000001
- type: precision_at_1 value: 82.97
- type: precision_at_10 value: 13.569
- type: precision_at_100 value: 1.539
- type: precision_at_1000 value: 0.157
- type: precision_at_3 value: 38.043
- type: precision_at_5 value: 24.992
- type: recall_at_1 value: 72.085
- type: recall_at_10 value: 96.262
- type: recall_at_100 value: 99.77000000000001
- type: recall_at_1000 value: 99.997
- type: recall_at_3 value: 88.652
- type: recall_at_5 value: 93.01899999999999 task: type: Retrieval
- dataset:
config: default
name: MTEB RedditClustering
revision: 24640382cdbf8abc73003fb0fa6d111a705499eb
split: test
type: mteb/reddit-clustering
metrics:
- type: v_measure value: 55.82153952668092 task: type: Clustering
- dataset:
config: default
name: MTEB RedditClusteringP2P
revision: 282350215ef01743dc01b456c7f5241fa8937f16
split: test
type: mteb/reddit-clustering-p2p
metrics:
- type: v_measure value: 62.094465801879295 task: type: Clustering
- dataset:
config: default
name: MTEB SCIDOCS
revision: None
split: test
type: mteb/scidocs
metrics:
- type: map_at_1 value: 5.688
- type: map_at_10 value: 15.201999999999998
- type: map_at_100 value: 18.096
- type: map_at_1000 value: 18.481
- type: map_at_3 value: 10.734
- type: map_at_5 value: 12.94
- type: mrr_at_1 value: 28.000000000000004
- type: mrr_at_10 value: 41.101
- type: mrr_at_100 value: 42.202
- type: mrr_at_1000 value: 42.228
- type: mrr_at_3 value: 37.683
- type: mrr_at_5 value: 39.708
- type: ndcg_at_1 value: 28.000000000000004
- type: ndcg_at_10 value: 24.976000000000003
- type: ndcg_at_100 value: 35.129
- type: ndcg_at_1000 value: 40.77
- type: ndcg_at_3 value: 23.787
- type: ndcg_at_5 value: 20.816000000000003
- type: precision_at_1 value: 28.000000000000004
- type: precision_at_10 value: 13.04
- type: precision_at_100 value: 2.761
- type: precision_at_1000 value: 0.41000000000000003
- type: precision_at_3 value: 22.6
- type: precision_at_5 value: 18.52
- type: recall_at_1 value: 5.688
- type: recall_at_10 value: 26.43
- type: recall_at_100 value: 56.02
- type: recall_at_1000 value: 83.21
- type: recall_at_3 value: 13.752
- type: recall_at_5 value: 18.777 task: type: Retrieval
- dataset:
config: default
name: MTEB SICK-R
revision: a6ea5a8cab320b040a23452cc28066d9beae2cee
split: test
type: mteb/sickr-sts
metrics:
- type: cos_sim_pearson value: 85.15084859283178
- type: cos_sim_spearman value: 80.49030614009419
- type: euclidean_pearson value: 81.84574978672468
- type: euclidean_spearman value: 79.89787150656818
- type: manhattan_pearson value: 81.63076538567131
- type: manhattan_spearman value: 79.69867352121841 task: type: STS
- dataset:
config: default
name: MTEB STS12
revision: a0d554a64d88156834ff5ae9920b964011b16384
split: test
type: mteb/sts12-sts
metrics:
- type: cos_sim_pearson value: 84.64097921490992
- type: cos_sim_spearman value: 77.25370084896514
- type: euclidean_pearson value: 82.71210826468788
- type: euclidean_spearman value: 78.50445584994826
- type: manhattan_pearson value: 82.92580164330298
- type: manhattan_spearman value: 78.69686891301019 task: type: STS
- dataset:
config: default
name: MTEB STS13
revision: 7e90230a92c190f1bf69ae9002b8cea547a64cca
split: test
type: mteb/sts13-sts
metrics:
- type: cos_sim_pearson value: 87.24596417308994
- type: cos_sim_spearman value: 87.79454220555091
- type: euclidean_pearson value: 87.40242561671164
- type: euclidean_spearman value: 88.25955597373556
- type: manhattan_pearson value: 87.25160240485849
- type: manhattan_spearman value: 88.155794979818 task: type: STS
- dataset:
config: default
name: MTEB STS14
revision: 6031580fec1f6af667f0bd2da0a551cf4f0b2375
split: test
type: mteb/sts14-sts
metrics:
- type: cos_sim_pearson value: 84.44914233422564
- type: cos_sim_spearman value: 82.91015471820322
- type: euclidean_pearson value: 84.7206656630327
- type: euclidean_spearman value: 83.86408872059216
- type: manhattan_pearson value: 84.72816725158454
- type: manhattan_spearman value: 84.01603388572788 task: type: STS
- dataset:
config: default
name: MTEB STS15
revision: ae752c7c21bf194d8b67fd573edf7ae58183cbe3
split: test
type: mteb/sts15-sts
metrics:
- type: cos_sim_pearson value: 87.6168026237477
- type: cos_sim_spearman value: 88.45414278092397
- type: euclidean_pearson value: 88.57023240882022
- type: euclidean_spearman value: 89.04102190922094
- type: manhattan_pearson value: 88.66695535796354
- type: manhattan_spearman value: 89.19898476680969 task: type: STS
- dataset:
config: default
name: MTEB STS16
revision: 4d8694f8f0e0100860b497b999b3dbed754a0513
split: test
type: mteb/sts16-sts
metrics:
- type: cos_sim_pearson value: 84.27925826089424
- type: cos_sim_spearman value: 85.45291099550461
- type: euclidean_pearson value: 83.63853036580834
- type: euclidean_spearman value: 84.33468035821484
- type: manhattan_pearson value: 83.72778773251596
- type: manhattan_spearman value: 84.51583132445376 task: type: STS
- dataset:
config: en-en
name: MTEB STS17 (en-en)
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
split: test
type: mteb/sts17-crosslingual-sts
metrics:
- type: cos_sim_pearson value: 89.67375185692552
- type: cos_sim_spearman value: 90.32542469203855
- type: euclidean_pearson value: 89.63513717951847
- type: euclidean_spearman value: 89.87760271003745
- type: manhattan_pearson value: 89.28381452982924
- type: manhattan_spearman value: 89.53568197785721 task: type: STS
- dataset:
config: en
name: MTEB STS22 (en)
revision: eea2b4fe26a775864c896887d910b76a8098ad3f
split: test
type: mteb/sts22-crosslingual-sts
metrics:
- type: cos_sim_pearson value: 66.24644693819846
- type: cos_sim_spearman value: 66.09889420525377
- type: euclidean_pearson value: 63.72551583520747
- type: euclidean_spearman value: 63.01385470780679
- type: manhattan_pearson value: 64.09258157214097
- type: manhattan_spearman value: 63.080517752822594 task: type: STS
- dataset:
config: default
name: MTEB STSBenchmark
revision: b0fddb56ed78048fa8b90373c8a3cfc37b684831
split: test
type: mteb/stsbenchmark-sts
metrics:
- type: cos_sim_pearson value: 86.27321463839989
- type: cos_sim_spearman value: 86.37572865993327
- type: euclidean_pearson value: 86.36268020198149
- type: euclidean_spearman value: 86.31089339478922
- type: manhattan_pearson value: 86.4260445761947
- type: manhattan_spearman value: 86.45885895320457 task: type: STS
- dataset:
config: default
name: MTEB SciDocsRR
revision: d3c5e1fc0b855ab6097bf1cda04dd73947d7caab
split: test
type: mteb/scidocs-reranking
metrics:
- type: map value: 86.52456702387798
- type: mrr value: 96.34556529164372 task: type: Reranking
- dataset:
config: default
name: MTEB SciFact
revision: 0228b52cf27578f30900b9e5271d331663a030d7
split: test
type: mteb/scifact
metrics:
- type: map_at_1 value: 61.99400000000001
- type: map_at_10 value: 73.38799999999999
- type: map_at_100 value: 73.747
- type: map_at_1000 value: 73.75
- type: map_at_3 value: 70.04599999999999
- type: map_at_5 value: 72.095
- type: mrr_at_1 value: 65.0
- type: mrr_at_10 value: 74.42800000000001
- type: mrr_at_100 value: 74.722
- type: mrr_at_1000 value: 74.725
- type: mrr_at_3 value: 72.056
- type: mrr_at_5 value: 73.60600000000001
- type: ndcg_at_1 value: 65.0
- type: ndcg_at_10 value: 78.435
- type: ndcg_at_100 value: 79.922
- type: ndcg_at_1000 value: 80.00500000000001
- type: ndcg_at_3 value: 73.05199999999999
- type: ndcg_at_5 value: 75.98
- type: precision_at_1 value: 65.0
- type: precision_at_10 value: 10.5
- type: precision_at_100 value: 1.123
- type: precision_at_1000 value: 0.11299999999999999
- type: precision_at_3 value: 28.555999999999997
- type: precision_at_5 value: 19.0
- type: recall_at_1 value: 61.99400000000001
- type: recall_at_10 value: 92.72200000000001
- type: recall_at_100 value: 99.333
- type: recall_at_1000 value: 100.0
- type: recall_at_3 value: 78.739
- type: recall_at_5 value: 85.828 task: type: Retrieval
- dataset:
config: default
name: MTEB SprintDuplicateQuestions
revision: d66bd1f72af766a5cc4b0ca5e00c162f89e8cc46
split: test
type: mteb/sprintduplicatequestions-pairclassification
metrics:
- type: cos_sim_accuracy value: 99.79009900990098
- type: cos_sim_ap value: 95.3203137438653
- type: cos_sim_f1 value: 89.12386706948641
- type: cos_sim_precision value: 89.75659229208925
- type: cos_sim_recall value: 88.5
- type: dot_accuracy value: 99.67821782178218
- type: dot_ap value: 89.94069840000675
- type: dot_f1 value: 83.45902463549521
- type: dot_precision value: 83.9231547017189
- type: dot_recall value: 83.0
- type: euclidean_accuracy value: 99.78613861386138
- type: euclidean_ap value: 95.10648259135526
- type: euclidean_f1 value: 88.77338877338877
- type: euclidean_precision value: 92.42424242424242
- type: euclidean_recall value: 85.39999999999999
- type: manhattan_accuracy value: 99.7950495049505
- type: manhattan_ap value: 95.29987661320946
- type: manhattan_f1 value: 89.21313183949972
- type: manhattan_precision value: 93.14472252448314
- type: manhattan_recall value: 85.6
- type: max_accuracy value: 99.7950495049505
- type: max_ap value: 95.3203137438653
- type: max_f1 value: 89.21313183949972 task: type: PairClassification
- dataset:
config: default
name: MTEB StackExchangeClustering
revision: 6cbc1f7b2bc0622f2e39d2c77fa502909748c259
split: test
type: mteb/stackexchange-clustering
metrics:
- type: v_measure value: 67.65446577183913 task: type: Clustering
- dataset:
config: default
name: MTEB StackExchangeClusteringP2P
revision: 815ca46b2622cec33ccafc3735d572c266efdb44
split: test
type: mteb/stackexchange-clustering-p2p
metrics:
- type: v_measure value: 46.30749237193961 task: type: Clustering
- dataset:
config: default
name: MTEB StackOverflowDupQuestions
revision: e185fbe320c72810689fc5848eb6114e1ef5ec69
split: test
type: mteb/stackoverflowdupquestions-reranking
metrics:
- type: map value: 54.91481849959949
- type: mrr value: 55.853506175197346 task: type: Reranking
- dataset:
config: default
name: MTEB SummEval
revision: cda12ad7615edc362dbf25a00fdd61d3b1eaf93c
split: test
type: mteb/summeval
metrics:
- type: cos_sim_pearson value: 30.08196549170419
- type: cos_sim_spearman value: 31.16661390597077
- type: dot_pearson value: 29.892258410943466
- type: dot_spearman value: 30.51328811965085 task: type: Summarization
- dataset:
config: default
name: MTEB TRECCOVID
revision: None
split: test
type: mteb/trec-covid
metrics:
- type: map_at_1 value: 0.23900000000000002
- type: map_at_10 value: 2.173
- type: map_at_100 value: 14.24
- type: map_at_1000 value: 35.309000000000005
- type: map_at_3 value: 0.7100000000000001
- type: map_at_5 value: 1.163
- type: mrr_at_1 value: 92.0
- type: mrr_at_10 value: 96.0
- type: mrr_at_100 value: 96.0
- type: mrr_at_1000 value: 96.0
- type: mrr_at_3 value: 96.0
- type: mrr_at_5 value: 96.0
- type: ndcg_at_1 value: 90.0
- type: ndcg_at_10 value: 85.382
- type: ndcg_at_100 value: 68.03
- type: ndcg_at_1000 value: 61.021
- type: ndcg_at_3 value: 89.765
- type: ndcg_at_5 value: 88.444
- type: precision_at_1 value: 92.0
- type: precision_at_10 value: 88.0
- type: precision_at_100 value: 70.02000000000001
- type: precision_at_1000 value: 26.984
- type: precision_at_3 value: 94.0
- type: precision_at_5 value: 92.80000000000001
- type: recall_at_1 value: 0.23900000000000002
- type: recall_at_10 value: 2.313
- type: recall_at_100 value: 17.049
- type: recall_at_1000 value: 57.489999999999995
- type: recall_at_3 value: 0.737
- type: recall_at_5 value: 1.221 task: type: Retrieval
- dataset:
config: default
name: MTEB Touche2020
revision: a34f9a33db75fa0cbb21bb5cfc3dae8dc8bec93f
split: test
type: mteb/touche2020
metrics:
- type: map_at_1 value: 2.75
- type: map_at_10 value: 11.29
- type: map_at_100 value: 18.032999999999998
- type: map_at_1000 value: 19.746
- type: map_at_3 value: 6.555
- type: map_at_5 value: 8.706999999999999
- type: mrr_at_1 value: 34.694
- type: mrr_at_10 value: 50.55
- type: mrr_at_100 value: 51.659
- type: mrr_at_1000 value: 51.659
- type: mrr_at_3 value: 47.278999999999996
- type: mrr_at_5 value: 49.728
- type: ndcg_at_1 value: 32.653
- type: ndcg_at_10 value: 27.894000000000002
- type: ndcg_at_100 value: 39.769
- type: ndcg_at_1000 value: 51.495999999999995
- type: ndcg_at_3 value: 32.954
- type: ndcg_at_5 value: 31.502999999999997
- type: precision_at_1 value: 34.694
- type: precision_at_10 value: 23.265
- type: precision_at_100 value: 7.898
- type: precision_at_1000 value: 1.58
- type: precision_at_3 value: 34.694
- type: precision_at_5 value: 31.429000000000002
- type: recall_at_1 value: 2.75
- type: recall_at_10 value: 16.953
- type: recall_at_100 value: 48.68
- type: recall_at_1000 value: 85.18599999999999
- type: recall_at_3 value: 7.710999999999999
- type: recall_at_5 value: 11.484 task: type: Retrieval
- dataset:
config: default
name: MTEB ToxicConversationsClassification
revision: d7c0de2777da35d6aae2200a62c6e0e5af397c4c
split: test
type: mteb/toxic_conversations_50k
metrics:
- type: accuracy value: 82.66099999999999
- type: ap value: 25.555698090238337
- type: f1 value: 66.48402012461622 task: type: Classification
- dataset:
config: default
name: MTEB TweetSentimentExtractionClassification
revision: d604517c81ca91fe16a244d1248fc021f9ecee7a
split: test
type: mteb/tweet_sentiment_extraction
metrics:
- type: accuracy value: 72.94567062818335
- type: f1 value: 73.28139189595674 task: type: Classification
- dataset:
config: default
name: MTEB TwentyNewsgroupsClustering
revision: 6125ec4e24fa026cec8a478383ee943acfbd5449
split: test
type: mteb/twentynewsgroups-clustering
metrics:
- type: v_measure value: 49.581627240203474 task: type: Clustering
- dataset:
config: default
name: MTEB TwitterSemEval2015
revision: 70970daeab8776df92f5ea462b6173c0b46fd2d1
split: test
type: mteb/twittersemeval2015-pairclassification
metrics:
- type: cos_sim_accuracy value: 87.78089050485785
- type: cos_sim_ap value: 79.64487116574168
- type: cos_sim_f1 value: 72.46563021970964
- type: cos_sim_precision value: 70.62359128474831
- type: cos_sim_recall value: 74.40633245382587
- type: dot_accuracy value: 86.2609524944865
- type: dot_ap value: 75.513046857613
- type: dot_f1 value: 68.58213616489695
- type: dot_precision value: 65.12455516014235
- type: dot_recall value: 72.42744063324538
- type: euclidean_accuracy value: 87.6080348095607
- type: euclidean_ap value: 79.00204933649795
- type: euclidean_f1 value: 72.14495342605589
- type: euclidean_precision value: 69.85421299728193
- type: euclidean_recall value: 74.5910290237467
- type: manhattan_accuracy value: 87.59611372712642
- type: manhattan_ap value: 78.78523756706264
- type: manhattan_f1 value: 71.86499137718648
- type: manhattan_precision value: 67.39833641404806
- type: manhattan_recall value: 76.96569920844327
- type: max_accuracy value: 87.78089050485785
- type: max_ap value: 79.64487116574168
- type: max_f1 value: 72.46563021970964 task: type: PairClassification
- dataset:
config: default
name: MTEB TwitterURLCorpus
revision: 8b6510b0b1fa4e4c4f879467980e9be563ec1cdf
split: test
type: mteb/twitterurlcorpus-pairclassification
metrics:
- type: cos_sim_accuracy value: 89.98719292117825
- type: cos_sim_ap value: 87.58146137353202
- type: cos_sim_f1 value: 80.28543232369239
- type: cos_sim_precision value: 79.1735289714029
- type: cos_sim_recall value: 81.42901139513397
- type: dot_accuracy value: 88.9199363526992
- type: dot_ap value: 84.98499998630417
- type: dot_f1 value: 78.21951400757969
- type: dot_precision value: 75.58523624874336
- type: dot_recall value: 81.04404065291038
- type: euclidean_accuracy value: 89.77374160748244
- type: euclidean_ap value: 87.35151562835209
- type: euclidean_f1 value: 79.92160922940393
- type: euclidean_precision value: 76.88531587933979
- type: euclidean_recall value: 83.20757622420696
- type: manhattan_accuracy value: 89.72717041176699
- type: manhattan_ap value: 87.34065592142515
- type: manhattan_f1 value: 79.85603419187943
- type: manhattan_precision value: 77.82243332115455
- type: manhattan_recall value: 81.99876809362489
- type: max_accuracy value: 89.98719292117825
- type: max_ap value: 87.58146137353202
- type: max_f1 value: 80.28543232369239 task: type: PairClassification
- dataset:
config: default
name: MTEB AFQMC
revision: b44c3b011063adb25877c13823db83bb193913c4
split: validation
type: C-MTEB/AFQMC
metrics:
- type: cos_sim_pearson value: 53.45954203592337
- type: cos_sim_spearman value: 58.42154680418638
- type: euclidean_pearson value: 56.41543791722753
- type: euclidean_spearman value: 58.39328016640146
- type: manhattan_pearson value: 56.318510356833876
- type: manhattan_spearman value: 58.28423447818184 task: type: STS
- dataset:
config: default
name: MTEB ATEC
revision: 0f319b1142f28d00e055a6770f3f726ae9b7d865
split: test
type: C-MTEB/ATEC
metrics:
- type: cos_sim_pearson value: 50.78356460675945
- type: cos_sim_spearman value: 55.6530411663269
- type: euclidean_pearson value: 56.50763660417816
- type: euclidean_spearman value: 55.733823335669065
- type: manhattan_pearson value: 56.45323093512866
- type: manhattan_spearman value: 55.63248619032702 task: type: STS
- dataset:
config: zh
name: MTEB AmazonReviewsClassification (zh)
revision: 1399c76144fd37290681b995c656ef9b2e06e26d
split: test
type: mteb/amazon_reviews_multi
metrics:
- type: accuracy value: 47.209999999999994
- type: f1 value: 46.08892432018655 task: type: Classification
- dataset:
config: default
name: MTEB BQ
revision: e3dda5e115e487b39ec7e618c0c6a29137052a55
split: test
type: C-MTEB/BQ
metrics:
- type: cos_sim_pearson value: 70.25573992001478
- type: cos_sim_spearman value: 73.85247134951433
- type: euclidean_pearson value: 72.60033082168442
- type: euclidean_spearman value: 73.72445893756499
- type: manhattan_pearson value: 72.59932284620231
- type: manhattan_spearman value: 73.68002490614583 task: type: STS
- dataset:
config: default
name: MTEB CLSClusteringP2P
revision: 4b6227591c6c1a73bc76b1055f3b7f3588e72476
split: test
type: C-MTEB/CLSClusteringP2P
metrics:
- type: v_measure value: 45.21317724305628 task: type: Clustering
- dataset:
config: default
name: MTEB CLSClusteringS2S
revision: e458b3f5414b62b7f9f83499ac1f5497ae2e869f
split: test
type: C-MTEB/CLSClusteringS2S
metrics:
- type: v_measure value: 42.49825170976724 task: type: Clustering
- dataset:
config: default
name: MTEB CMedQAv1
revision: 8d7f1e942507dac42dc58017c1a001c3717da7df
split: test
type: C-MTEB/CMedQAv1-reranking
metrics:
- type: map value: 88.15661686810597
- type: mrr value: 90.11222222222223 task: type: Reranking
- dataset:
config: default
name: MTEB CMedQAv2
revision: 23d186750531a14a0357ca22cd92d712fd512ea0
split: test
type: C-MTEB/CMedQAv2-reranking
metrics:
- type: map value: 88.1204726064383
- type: mrr value: 90.20142857142858 task: type: Reranking
- dataset:
config: default
name: MTEB CmedqaRetrieval
revision: cd540c506dae1cf9e9a59c3e06f42030d54e7301
split: dev
type: C-MTEB/CmedqaRetrieval
metrics:
- type: map_at_1 value: 27.224999999999998
- type: map_at_10 value: 40.169
- type: map_at_100 value: 42.0
- type: map_at_1000 value: 42.109
- type: map_at_3 value: 35.76
- type: map_at_5 value: 38.221
- type: mrr_at_1 value: 40.56
- type: mrr_at_10 value: 49.118
- type: mrr_at_100 value: 50.092999999999996
- type: mrr_at_1000 value: 50.133
- type: mrr_at_3 value: 46.507
- type: mrr_at_5 value: 47.973
- type: ndcg_at_1 value: 40.56
- type: ndcg_at_10 value: 46.972
- type: ndcg_at_100 value: 54.04
- type: ndcg_at_1000 value: 55.862
- type: ndcg_at_3 value: 41.36
- type: ndcg_at_5 value: 43.704
- type: precision_at_1 value: 40.56
- type: precision_at_10 value: 10.302999999999999
- type: precision_at_100 value: 1.606
- type: precision_at_1000 value: 0.184
- type: precision_at_3 value: 23.064
- type: precision_at_5 value: 16.764000000000003
- type: recall_at_1 value: 27.224999999999998
- type: recall_at_10 value: 58.05200000000001
- type: recall_at_100 value: 87.092
- type: recall_at_1000 value: 99.099
- type: recall_at_3 value: 41.373
- type: recall_at_5 value: 48.453 task: type: Retrieval
- dataset:
config: default
name: MTEB Cmnli
revision: 41bc36f332156f7adc9e38f53777c959b2ae9766
split: validation
type: C-MTEB/CMNLI
metrics:
- type: cos_sim_accuracy value: 77.40228502705953
- type: cos_sim_ap value: 86.22359172956327
- type: cos_sim_f1 value: 78.96328293736501
- type: cos_sim_precision value: 73.36945615091311
- type: cos_sim_recall value: 85.48047696983868
- type: dot_accuracy value: 75.53818400481059
- type: dot_ap value: 83.70164011305312
- type: dot_f1 value: 77.67298719348754
- type: dot_precision value: 67.49482401656314
- type: dot_recall value: 91.46598082768296
- type: euclidean_accuracy value: 77.94347564642213
- type: euclidean_ap value: 86.4652108728609
- type: euclidean_f1 value: 79.15555555555555
- type: euclidean_precision value: 75.41816641964853
- type: euclidean_recall value: 83.28267477203647
- type: manhattan_accuracy value: 77.45039085989175
- type: manhattan_ap value: 86.09986583900665
- type: manhattan_f1 value: 78.93669264438988
- type: manhattan_precision value: 72.63261296660117
- type: manhattan_recall value: 86.43909282207154
- type: max_accuracy value: 77.94347564642213
- type: max_ap value: 86.4652108728609
- type: max_f1 value: 79.15555555555555 task: type: PairClassification
- dataset:
config: default
name: MTEB CovidRetrieval
revision: 1271c7809071a13532e05f25fb53511ffce77117
split: dev
type: C-MTEB/CovidRetrieval
metrics:
- type: map_at_1 value: 69.336
- type: map_at_10 value: 77.16
- type: map_at_100 value: 77.47500000000001
- type: map_at_1000 value: 77.482
- type: map_at_3 value: 75.42999999999999
- type: map_at_5 value: 76.468
- type: mrr_at_1 value: 69.44200000000001
- type: mrr_at_10 value: 77.132
- type: mrr_at_100 value: 77.43299999999999
- type: mrr_at_1000 value: 77.44
- type: mrr_at_3 value: 75.395
- type: mrr_at_5 value: 76.459
- type: ndcg_at_1 value: 69.547
- type: ndcg_at_10 value: 80.794
- type: ndcg_at_100 value: 82.245
- type: ndcg_at_1000 value: 82.40899999999999
- type: ndcg_at_3 value: 77.303
- type: ndcg_at_5 value: 79.168
- type: precision_at_1 value: 69.547
- type: precision_at_10 value: 9.305
- type: precision_at_100 value: 0.9979999999999999
- type: precision_at_1000 value: 0.101
- type: precision_at_3 value: 27.749000000000002
- type: precision_at_5 value: 17.576
- type: recall_at_1 value: 69.336
- type: recall_at_10 value: 92.097
- type: recall_at_100 value: 98.736
- type: recall_at_1000 value: 100.0
- type: recall_at_3 value: 82.64
- type: recall_at_5 value: 87.144 task: type: Retrieval
- dataset:
config: default
name: MTEB DuRetrieval
revision: a1a333e290fe30b10f3f56498e3a0d911a693ced
split: dev
type: C-MTEB/DuRetrieval
metrics:
- type: map_at_1 value: 26.817999999999998
- type: map_at_10 value: 82.67
- type: map_at_100 value: 85.304
- type: map_at_1000 value: 85.334
- type: map_at_3 value: 57.336
- type: map_at_5 value: 72.474
- type: mrr_at_1 value: 91.45
- type: mrr_at_10 value: 94.272
- type: mrr_at_100 value: 94.318
- type: mrr_at_1000 value: 94.32000000000001
- type: mrr_at_3 value: 94.0
- type: mrr_at_5 value: 94.17699999999999
- type: ndcg_at_1 value: 91.45
- type: ndcg_at_10 value: 89.404
- type: ndcg_at_100 value: 91.724
- type: ndcg_at_1000 value: 91.973
- type: ndcg_at_3 value: 88.104
- type: ndcg_at_5 value: 87.25699999999999
- type: precision_at_1 value: 91.45
- type: precision_at_10 value: 42.585
- type: precision_at_100 value: 4.838
- type: precision_at_1000 value: 0.49
- type: precision_at_3 value: 78.8
- type: precision_at_5 value: 66.66
- type: recall_at_1 value: 26.817999999999998
- type: recall_at_10 value: 90.67
- type: recall_at_100 value: 98.36200000000001
- type: recall_at_1000 value: 99.583
- type: recall_at_3 value: 59.614999999999995
- type: recall_at_5 value: 77.05199999999999 task: type: Retrieval
- dataset:
config: default
name: MTEB EcomRetrieval
revision: 687de13dc7294d6fd9be10c6945f9e8fec8166b9
split: dev
type: C-MTEB/EcomRetrieval
metrics:
- type: map_at_1 value: 47.699999999999996
- type: map_at_10 value: 57.589999999999996
- type: map_at_100 value: 58.226
- type: map_at_1000 value: 58.251
- type: map_at_3 value: 55.233
- type: map_at_5 value: 56.633
- type: mrr_at_1 value: 47.699999999999996
- type: mrr_at_10 value: 57.589999999999996
- type: mrr_at_100 value: 58.226
- type: mrr_at_1000 value: 58.251
- type: mrr_at_3 value: 55.233
- type: mrr_at_5 value: 56.633
- type: ndcg_at_1 value: 47.699999999999996
- type: ndcg_at_10 value: 62.505
- type: ndcg_at_100 value: 65.517
- type: ndcg_at_1000 value: 66.19800000000001
- type: ndcg_at_3 value: 57.643
- type: ndcg_at_5 value: 60.181
- type: precision_at_1 value: 47.699999999999996
- type: precision_at_10 value: 7.8
- type: precision_at_100 value: 0.919
- type: precision_at_1000 value: 0.097
- type: precision_at_3 value: 21.532999999999998
- type: precision_at_5 value: 14.16
- type: recall_at_1 value: 47.699999999999996
- type: recall_at_10 value: 78.0
- type: recall_at_100 value: 91.9
- type: recall_at_1000 value: 97.3
- type: recall_at_3 value: 64.60000000000001
- type: recall_at_5 value: 70.8 task: type: Retrieval
- dataset:
config: default
name: MTEB IFlyTek
revision: 421605374b29664c5fc098418fe20ada9bd55f8a
split: validation
type: C-MTEB/IFlyTek-classification
metrics:
- type: accuracy value: 44.84801846864178
- type: f1 value: 37.47347897956339 task: type: Classification
- dataset:
config: default
name: MTEB JDReview
revision: b7c64bd89eb87f8ded463478346f76731f07bf8b
split: test
type: C-MTEB/JDReview-classification
metrics:
- type: accuracy value: 85.81613508442777
- type: ap value: 52.68244615477374
- type: f1 value: 80.0445640948843 task: type: Classification
- dataset:
config: default
name: MTEB LCQMC
revision: 17f9b096f80380fce5ed12a9be8be7784b337daf
split: test
type: C-MTEB/LCQMC
metrics:
- type: cos_sim_pearson value: 69.57786502217138
- type: cos_sim_spearman value: 75.39106054489906
- type: euclidean_pearson value: 73.72082954602402
- type: euclidean_spearman value: 75.14421475913619
- type: manhattan_pearson value: 73.62463076633642
- type: manhattan_spearman value: 75.01301565104112 task: type: STS
- dataset:
config: default
name: MTEB MMarcoReranking
revision: None
split: dev
type: C-MTEB/Mmarco-reranking
metrics:
- type: map value: 29.143797057999134
- type: mrr value: 28.08174603174603 task: type: Reranking
- dataset:
config: default
name: MTEB MMarcoRetrieval
revision: 539bbde593d947e2a124ba72651aafc09eb33fc2
split: dev
type: C-MTEB/MMarcoRetrieval
metrics:
- type: map_at_1 value: 70.492
- type: map_at_10 value: 79.501
- type: map_at_100 value: 79.728
- type: map_at_1000 value: 79.735
- type: map_at_3 value: 77.77
- type: map_at_5 value: 78.851
- type: mrr_at_1 value: 72.822
- type: mrr_at_10 value: 80.001
- type: mrr_at_100 value: 80.19
- type: mrr_at_1000 value: 80.197
- type: mrr_at_3 value: 78.484
- type: mrr_at_5 value: 79.42099999999999
- type: ndcg_at_1 value: 72.822
- type: ndcg_at_10 value: 83.013
- type: ndcg_at_100 value: 84.013
- type: ndcg_at_1000 value: 84.20400000000001
- type: ndcg_at_3 value: 79.728
- type: ndcg_at_5 value: 81.542
- type: precision_at_1 value: 72.822
- type: precision_at_10 value: 9.917
- type: precision_at_100 value: 1.042
- type: precision_at_1000 value: 0.106
- type: precision_at_3 value: 29.847
- type: precision_at_5 value: 18.871
- type: recall_at_1 value: 70.492
- type: recall_at_10 value: 93.325
- type: recall_at_100 value: 97.822
- type: recall_at_1000 value: 99.319
- type: recall_at_3 value: 84.636
- type: recall_at_5 value: 88.93100000000001 task: type: Retrieval
- dataset:
config: zh-CN
name: MTEB MassiveIntentClassification (zh-CN)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy value: 76.88298587760592
- type: f1 value: 73.89001762017176 task: type: Classification
- dataset:
config: zh-CN
name: MTEB MassiveScenarioClassification (zh-CN)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy value: 80.76328177538669
- type: f1 value: 80.24718532423358 task: type: Classification
- dataset:
config: default
name: MTEB MedicalRetrieval
revision: 2039188fb5800a9803ba5048df7b76e6fb151fc6
split: dev
type: C-MTEB/MedicalRetrieval
metrics:
- type: map_at_1 value: 49.6
- type: map_at_10 value: 55.620999999999995
- type: map_at_100 value: 56.204
- type: map_at_1000 value: 56.251
- type: map_at_3 value: 54.132999999999996
- type: map_at_5 value: 54.933
- type: mrr_at_1 value: 49.7
- type: mrr_at_10 value: 55.67100000000001
- type: mrr_at_100 value: 56.254000000000005
- type: mrr_at_1000 value: 56.301
- type: mrr_at_3 value: 54.18300000000001
- type: mrr_at_5 value: 54.983000000000004
- type: ndcg_at_1 value: 49.6
- type: ndcg_at_10 value: 58.645
- type: ndcg_at_100 value: 61.789
- type: ndcg_at_1000 value: 63.219
- type: ndcg_at_3 value: 55.567
- type: ndcg_at_5 value: 57.008
- type: precision_at_1 value: 49.6
- type: precision_at_10 value: 6.819999999999999
- type: precision_at_100 value: 0.836
- type: precision_at_1000 value: 0.095
- type: precision_at_3 value: 19.900000000000002
- type: precision_at_5 value: 12.64
- type: recall_at_1 value: 49.6
- type: recall_at_10 value: 68.2
- type: recall_at_100 value: 83.6
- type: recall_at_1000 value: 95.3
- type: recall_at_3 value: 59.699999999999996
- type: recall_at_5 value: 63.2 task: type: Retrieval
- dataset:
config: default
name: MTEB MultilingualSentiment
revision: 46958b007a63fdbf239b7672c25d0bea67b5ea1a
split: validation
type: C-MTEB/MultilingualSentiment-classification
metrics:
- type: accuracy value: 74.45666666666666
- type: f1 value: 74.32582402190089 task: type: Classification
- dataset:
config: default
name: MTEB Ocnli
revision: 66e76a618a34d6d565d5538088562851e6daa7ec
split: validation
type: C-MTEB/OCNLI
metrics:
- type: cos_sim_accuracy value: 80.67135896047645
- type: cos_sim_ap value: 87.60421240712051
- type: cos_sim_f1 value: 82.1304131408661
- type: cos_sim_precision value: 77.68361581920904
- type: cos_sim_recall value: 87.11721224920802
- type: dot_accuracy value: 79.04710341093666
- type: dot_ap value: 85.6370059719336
- type: dot_f1 value: 80.763723150358
- type: dot_precision value: 73.69337979094077
- type: dot_recall value: 89.33474128827878
- type: euclidean_accuracy value: 81.05035192203573
- type: euclidean_ap value: 87.7880240053663
- type: euclidean_f1 value: 82.50244379276637
- type: euclidean_precision value: 76.7970882620564
- type: euclidean_recall value: 89.1235480464625
- type: manhattan_accuracy value: 80.61721710882512
- type: manhattan_ap value: 87.43568120591175
- type: manhattan_f1 value: 81.89526184538653
- type: manhattan_precision value: 77.5992438563327
- type: manhattan_recall value: 86.6948257655755
- type: max_accuracy value: 81.05035192203573
- type: max_ap value: 87.7880240053663
- type: max_f1 value: 82.50244379276637 task: type: PairClassification
- dataset:
config: default
name: MTEB OnlineShopping
revision: e610f2ebd179a8fda30ae534c3878750a96db120
split: test
type: C-MTEB/OnlineShopping-classification
metrics:
- type: accuracy value: 93.5
- type: ap value: 91.31357903446782
- type: f1 value: 93.48088994006616 task: type: Classification
- dataset:
config: default
name: MTEB PAWSX
revision: 9c6a90e430ac22b5779fb019a23e820b11a8b5e1
split: test
type: C-MTEB/PAWSX
metrics:
- type: cos_sim_pearson value: 36.93293453538077
- type: cos_sim_spearman value: 42.45972506308574
- type: euclidean_pearson value: 42.34945133152159
- type: euclidean_spearman value: 42.331610303674644
- type: manhattan_pearson value: 42.31455070249498
- type: manhattan_spearman value: 42.19887982891834 task: type: STS
- dataset:
config: default
name: MTEB QBQTC
revision: 790b0510dc52b1553e8c49f3d2afb48c0e5c48b7
split: test
type: C-MTEB/QBQTC
metrics:
- type: cos_sim_pearson value: 33.683290790043785
- type: cos_sim_spearman value: 35.149171171202994
- type: euclidean_pearson value: 32.33806561267862
- type: euclidean_spearman value: 34.483576387347966
- type: manhattan_pearson value: 32.47629754599608
- type: manhattan_spearman value: 34.66434471867615 task: type: STS
- dataset:
config: zh
name: MTEB STS22 (zh)
revision: eea2b4fe26a775864c896887d910b76a8098ad3f
split: test
type: mteb/sts22-crosslingual-sts
metrics:
- type: cos_sim_pearson value: 66.46322760516104
- type: cos_sim_spearman value: 67.398478319726
- type: euclidean_pearson value: 64.7223480293625
- type: euclidean_spearman value: 66.83118568812951
- type: manhattan_pearson value: 64.88440039828305
- type: manhattan_spearman value: 66.80429458952257 task: type: STS
- dataset:
config: default
name: MTEB STSB
revision: 0cde68302b3541bb8b3c340dc0644b0b745b3dc0
split: test
type: C-MTEB/STSB
metrics:
- type: cos_sim_pearson value: 79.08991383232105
- type: cos_sim_spearman value: 79.39715677296854
- type: euclidean_pearson value: 78.63201279320496
- type: euclidean_spearman value: 79.40262660785731
- type: manhattan_pearson value: 78.98138363146906
- type: manhattan_spearman value: 79.79968413014194 task: type: STS
- dataset:
config: default
name: MTEB T2Reranking
revision: 76631901a18387f85eaa53e5450019b87ad58ef9
split: dev
type: C-MTEB/T2Reranking
metrics:
- type: map value: 67.43289278789972
- type: mrr value: 77.53012460908535 task: type: Reranking
- dataset:
config: default
name: MTEB T2Retrieval
revision: 8731a845f1bf500a4f111cf1070785c793d10e64
split: dev
type: C-MTEB/T2Retrieval
metrics:
- type: map_at_1 value: 27.733999999999998
- type: map_at_10 value: 78.24799999999999
- type: map_at_100 value: 81.765
- type: map_at_1000 value: 81.824
- type: map_at_3 value: 54.92
- type: map_at_5 value: 67.61399999999999
- type: mrr_at_1 value: 90.527
- type: mrr_at_10 value: 92.843
- type: mrr_at_100 value: 92.927
- type: mrr_at_1000 value: 92.93
- type: mrr_at_3 value: 92.45100000000001
- type: mrr_at_5 value: 92.693
- type: ndcg_at_1 value: 90.527
- type: ndcg_at_10 value: 85.466
- type: ndcg_at_100 value: 88.846
- type: ndcg_at_1000 value: 89.415
- type: ndcg_at_3 value: 86.768
- type: ndcg_at_5 value: 85.46000000000001
- type: precision_at_1 value: 90.527
- type: precision_at_10 value: 42.488
- type: precision_at_100 value: 5.024
- type: precision_at_1000 value: 0.516
- type: precision_at_3 value: 75.907
- type: precision_at_5 value: 63.727000000000004
- type: recall_at_1 value: 27.733999999999998
- type: recall_at_10 value: 84.346
- type: recall_at_100 value: 95.536
- type: recall_at_1000 value: 98.42999999999999
- type: recall_at_3 value: 56.455
- type: recall_at_5 value: 70.755 task: type: Retrieval
- dataset:
config: default
name: MTEB TNews
revision: 317f262bf1e6126357bbe89e875451e4b0938fe4
split: validation
type: C-MTEB/TNews-classification
metrics:
- type: accuracy value: 49.952000000000005
- type: f1 value: 48.264617195258054 task: type: Classification
- dataset:
config: default
name: MTEB ThuNewsClusteringP2P
revision: 5798586b105c0434e4f0fe5e767abe619442cf93
split: test
type: C-MTEB/ThuNewsClusteringP2P
metrics:
- type: v_measure value: 68.23769904483508 task: type: Clustering
- dataset:
config: default
name: MTEB ThuNewsClusteringS2S
revision: 8a8b2caeda43f39e13c4bc5bea0f8a667896e10d
split: test
type: C-MTEB/ThuNewsClusteringS2S
metrics:
- type: v_measure value: 62.50294403136556 task: type: Clustering
- dataset:
config: default
name: MTEB VideoRetrieval
revision: 58c2597a5943a2ba48f4668c3b90d796283c5639
split: dev
type: C-MTEB/VideoRetrieval
metrics:
- type: map_at_1 value: 54.0
- type: map_at_10 value: 63.668
- type: map_at_100 value: 64.217
- type: map_at_1000 value: 64.23100000000001
- type: map_at_3 value: 61.7
- type: map_at_5 value: 62.870000000000005
- type: mrr_at_1 value: 54.0
- type: mrr_at_10 value: 63.668
- type: mrr_at_100 value: 64.217
- type: mrr_at_1000 value: 64.23100000000001
- type: mrr_at_3 value: 61.7
- type: mrr_at_5 value: 62.870000000000005
- type: ndcg_at_1 value: 54.0
- type: ndcg_at_10 value: 68.11399999999999
- type: ndcg_at_100 value: 70.723
- type: ndcg_at_1000 value: 71.123
- type: ndcg_at_3 value: 64.074
- type: ndcg_at_5 value: 66.178
- type: precision_at_1 value: 54.0
- type: precision_at_10 value: 8.200000000000001
- type: precision_at_100 value: 0.941
- type: precision_at_1000 value: 0.097
- type: precision_at_3 value: 23.633000000000003
- type: precision_at_5 value: 15.2
- type: recall_at_1 value: 54.0
- type: recall_at_10 value: 82.0
- type: recall_at_100 value: 94.1
- type: recall_at_1000 value: 97.3
- type: recall_at_3 value: 70.89999999999999
- type: recall_at_5 value: 76.0 task: type: Retrieval
- dataset:
config: default
name: MTEB Waimai
revision: 339287def212450dcaa9df8c22bf93e9980c7023
split: test
type: C-MTEB/waimai-classification
metrics:
- type: accuracy value: 86.63000000000001
- type: ap value: 69.99457882599567
- type: f1 value: 85.07735617998541 task: type: Classification
- dataset:
config: default
name: MTEB 8TagsClustering
revision: None
split: test
type: PL-MTEB/8tags-clustering
metrics:
- type: v_measure value: 44.594104491193555 task: type: Clustering
- dataset:
config: default
name: MTEB AllegroReviews
revision: None
split: test
type: PL-MTEB/allegro-reviews
metrics:
- type: accuracy value: 63.97614314115309
- type: f1 value: 52.15634261679283 task: type: Classification
- dataset:
config: default
name: MTEB ArguAna-PL
revision: 63fc86750af76253e8c760fc9e534bbf24d260a2
split: test
type: clarin-knext/arguana-pl
metrics:
- type: map_at_1 value: 32.646
- type: map_at_10 value: 47.963
- type: map_at_100 value: 48.789
- type: map_at_1000 value: 48.797000000000004
- type: map_at_3 value: 43.196
- type: map_at_5 value: 46.016
- type: mrr_at_1 value: 33.073
- type: mrr_at_10 value: 48.126000000000005
- type: mrr_at_100 value: 48.946
- type: mrr_at_1000 value: 48.953
- type: mrr_at_3 value: 43.374
- type: mrr_at_5 value: 46.147
- type: ndcg_at_1 value: 32.646
- type: ndcg_at_10 value: 56.481
- type: ndcg_at_100 value: 59.922
- type: ndcg_at_1000 value: 60.07
- type: ndcg_at_3 value: 46.675
- type: ndcg_at_5 value: 51.76500000000001
- type: precision_at_1 value: 32.646
- type: precision_at_10 value: 8.371
- type: precision_at_100 value: 0.9860000000000001
- type: precision_at_1000 value: 0.1
- type: precision_at_3 value: 18.919
- type: precision_at_5 value: 13.825999999999999
- type: recall_at_1 value: 32.646
- type: recall_at_10 value: 83.71300000000001
- type: recall_at_100 value: 98.578
- type: recall_at_1000 value: 99.644
- type: recall_at_3 value: 56.757000000000005
- type: recall_at_5 value: 69.132 task: type: Retrieval
- dataset:
config: default
name: MTEB CBD
revision: None
split: test
type: PL-MTEB/cbd
metrics:
- type: accuracy value: 68.56
- type: ap value: 23.310493680488513
- type: f1 value: 58.85369533105693 task: type: Classification
- dataset:
config: default
name: MTEB CDSC-E
revision: None
split: test
type: PL-MTEB/cdsce-pairclassification
metrics:
- type: cos_sim_accuracy value: 88.5
- type: cos_sim_ap value: 72.42140924378361
- type: cos_sim_f1 value: 66.0919540229885
- type: cos_sim_precision value: 72.78481012658227
- type: cos_sim_recall value: 60.526315789473685
- type: dot_accuracy value: 88.5
- type: dot_ap value: 72.42140924378361
- type: dot_f1 value: 66.0919540229885
- type: dot_precision value: 72.78481012658227
- type: dot_recall value: 60.526315789473685
- type: euclidean_accuracy value: 88.5
- type: euclidean_ap value: 72.42140924378361
- type: euclidean_f1 value: 66.0919540229885
- type: euclidean_precision value: 72.78481012658227
- type: euclidean_recall value: 60.526315789473685
- type: manhattan_accuracy value: 88.5
- type: manhattan_ap value: 72.49745515311696
- type: manhattan_f1 value: 66.0968660968661
- type: manhattan_precision value: 72.04968944099379
- type: manhattan_recall value: 61.05263157894737
- type: max_accuracy value: 88.5
- type: max_ap value: 72.49745515311696
- type: max_f1 value: 66.0968660968661 task: type: PairClassification
- dataset:
config: default
name: MTEB CDSC-R
revision: None
split: test
type: PL-MTEB/cdscr-sts
metrics:
- type: cos_sim_pearson value: 90.32269765590145
- type: cos_sim_spearman value: 89.73666311491672
- type: euclidean_pearson value: 88.2933868516544
- type: euclidean_spearman value: 89.73666311491672
- type: manhattan_pearson value: 88.33474590219448
- type: manhattan_spearman value: 89.8548364866583 task: type: STS
- dataset:
config: default
name: MTEB DBPedia-PL
revision: 76afe41d9af165cc40999fcaa92312b8b012064a
split: test
type: clarin-knext/dbpedia-pl
metrics:
- type: map_at_1 value: 7.632999999999999
- type: map_at_10 value: 16.426
- type: map_at_100 value: 22.651
- type: map_at_1000 value: 24.372
- type: map_at_3 value: 11.706
- type: map_at_5 value: 13.529
- type: mrr_at_1 value: 60.75000000000001
- type: mrr_at_10 value: 68.613
- type: mrr_at_100 value: 69.001
- type: mrr_at_1000 value: 69.021
- type: mrr_at_3 value: 67.0
- type: mrr_at_5 value: 67.925
- type: ndcg_at_1 value: 49.875
- type: ndcg_at_10 value: 36.978
- type: ndcg_at_100 value: 40.031
- type: ndcg_at_1000 value: 47.566
- type: ndcg_at_3 value: 41.148
- type: ndcg_at_5 value: 38.702
- type: precision_at_1 value: 60.75000000000001
- type: precision_at_10 value: 29.7
- type: precision_at_100 value: 9.278
- type: precision_at_1000 value: 2.099
- type: precision_at_3 value: 44.0
- type: precision_at_5 value: 37.6
- type: recall_at_1 value: 7.632999999999999
- type: recall_at_10 value: 22.040000000000003
- type: recall_at_100 value: 44.024
- type: recall_at_1000 value: 67.848
- type: recall_at_3 value: 13.093
- type: recall_at_5 value: 15.973 task: type: Retrieval
- dataset:
config: default
name: MTEB FiQA-PL
revision: 2e535829717f8bf9dc829b7f911cc5bbd4e6608e
split: test
type: clarin-knext/fiqa-pl
metrics:
- type: map_at_1
- dataset:
config: en
name: MTEB AmazonCounterfactualClassification (en)
revision: e8379541af4e31359cca9fbcf4b00f2671dba205
split: test
type: mteb/amazon_counterfactual
metrics:
The model card is truncated. Read the rest on Hugging Face.
- Created
- Jul 14, 2024
- Updated
- Jul 14, 2024