Downloads · 30 days
11
19% of all-time downloads
skfrost19/reranker-msmarco-v1.1-Lion-all-MiniLM-L6-v2-bce
reranker-msmarco-v1.1-Lion-all-MiniLM-L6-v2-bce is a text ranking model from skfrost19. Use it for the text ranking task on the model card, and read the license before you ship it in a product. It is set up for sentence-transformers.
This is a Cross Encoder model finetuned from sentence-transformers/all-MiniLM-L6-v2 on the msmarco dataset using the sentence-transformers library. It computes scores for pairs of texts, which can be used for text rer…
Downloads · 30 days
11
19% of all-time downloads
All-time downloads
57
Public
Parameters
22.7M
90.9 MB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors90.9 MB · 99%
From the Hugging Face model README
This is a Cross Encoder model finetuned from sentence-transformers/all-MiniLM-L6-v2 on the ms_marco dataset using the sentence-transformers library. It computes scores for pairs of texts, which can be used for text reranking and semantic search.
First install the Sentence Transformers library:
pip install -U sentence-transformers
Then you can load this model and run inference.
from sentence_transformers import CrossEncoder
# Download from the 🤗 Hub
model = CrossEncoder("skfrost19/reranker-msmarco-v1.1-Lion-all-MiniLM-L6-v2-bce")
# Get scores for pairs of texts
pairs = [
['what is a ladyfinger', 'A light, delicate sponge cake roughly shaped like a large, fat finger. Used as an accompaniment to ice cream, puddings and other desserts. One of the oldest and most delicate of sponge cakes, from the House of Savoy in eleventh century France. American ladyfingers are smaller and moister than their Italian counterparts.'],
['what is itp blood disorder', "Immune thrombocytopenia (THROM-bo-si-toe-PE-ne-ah), or ITP, is a bleeding disorder. In ITP, the blood doesn't clot as it should. This is due to a low number of blood cell fragments called platelets (PLATE-lets) or thrombocytes (THROM-bo-sites). Platelets are made in your bone marrow along with other kinds of blood cells."],
['what is spark programming', 'Spark 1.5.1 works with Java 7 and higher. If you are using Java 8, Spark supports lambda expressions for concisely writing functions, otherwise you can use the classes in the org.apache.spark.api.java.function package. To write a Spark application in Java, you need to add a dependency on Spark. The first thing a Spark program must do is to create a JavaSparkContext object, which tells Spark how to access a cluster. To create a SparkContext you first need to build a SparkConf object that contains information about your application.'],
['cost to replace a driveway', "The average national cost of a driveway installation is $3,647, with most homeowners spending between $2,026 and $5,278. This data is based on actual project costs as reported by HomeAdvisor members. If you're thinking about installing a driveway, it's important to consider a couple of different things. "],
['is quaker oatmeal gluten-free', 'Gluten-Free Confidence Score: 5/10. While pure oats themselves are technically gluten-free, cross contamination can occur while growing, storing, and processing of oats. It is because of this that Quaker Oats cannot guarantee that their oats in their oatmeal are truly gluten-free.'],
]
scores = model.predict(pairs)
print(scores.shape)
# (5,)
# Or rank different texts based on similarity to a single text
ranks = model.rank(
'what is a ladyfinger',
[
'A light, delicate sponge cake roughly shaped like a large, fat finger. Used as an accompaniment to ice cream, puddings and other desserts. One of the oldest and most delicate of sponge cakes, from the House of Savoy in eleventh century France. American ladyfingers are smaller and moister than their Italian counterparts.',
"Immune thrombocytopenia (THROM-bo-si-toe-PE-ne-ah), or ITP, is a bleeding disorder. In ITP, the blood doesn't clot as it should. This is due to a low number of blood cell fragments called platelets (PLATE-lets) or thrombocytes (THROM-bo-sites). Platelets are made in your bone marrow along with other kinds of blood cells.",
'Spark 1.5.1 works with Java 7 and higher. If you are using Java 8, Spark supports lambda expressions for concisely writing functions, otherwise you can use the classes in the org.apache.spark.api.java.function package. To write a Spark application in Java, you need to add a dependency on Spark. The first thing a Spark program must do is to create a JavaSparkContext object, which tells Spark how to access a cluster. To create a SparkContext you first need to build a SparkConf object that contains information about your application.',
"The average national cost of a driveway installation is $3,647, with most homeowners spending between $2,026 and $5,278. This data is based on actual project costs as reported by HomeAdvisor members. If you're thinking about installing a driveway, it's important to consider a couple of different things. ",
'Gluten-Free Confidence Score: 5/10. While pure oats themselves are technically gluten-free, cross contamination can occur while growing, storing, and processing of oats. It is because of this that Quaker Oats cannot guarantee that their oats in their oatmeal are truly gluten-free.',
]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]
<!--
### Direct Usage (Transformers)
<details><summary>Click to see the direct usage in Transformers</summary>
</details>
-->
<!--
### Downstream Usage (Sentence Transformers)
You can finetune this model on your own dataset.
<details><summary>Click to expand</summary>
</details>
-->
<!--
### Out-of-Scope Use
*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->
NanoMSMARCO_R100, NanoNFCorpus_R100 and NanoNQ_R100{
"at_k": 10,
"always_rerank_positives": true
}
| Metric | NanoMSMARCO_R100 | NanoNFCorpus_R100 | NanoNQ_R100 |
|---|---|---|---|
| map | 0.0637 (-0.4259) | 0.2746 (+0.0136) | 0.0348 (-0.3848) |
| mrr@10 | 0.0361 (-0.4414) | 0.2927 (-0.2071) | 0.0111 (-0.4156) |
| ndcg@10 | 0.0507 (-0.4898) | 0.2176 (-0.1074) | 0.0300 (-0.4706) |
NanoBEIR_R100_mean{
"dataset_names": [
"msmarco",
"nfcorpus",
"nq"
],
"rerank_k": 100,
"at_k": 10,
"always_rerank_positives": true
}
| Metric | Value |
|---|---|
| map | 0.1244 (-0.2657) |
| mrr@10 | 0.1133 (-0.3547) |
| ndcg@10 | 0.0994 (-0.3559) |
| query | passage | label | |
|---|---|---|---|
| type | string | string | int |
| details | <ul><li>min: 8 characters</li><li>mean: 33.72 characters</li><li>max: 101 characters</li></ul> | <ul><li>min: 121 characters</li><li>mean: 417.75 characters</li><li>max: 843 characters</li></ul> | <ul><li>0: ~86.60%</li><li>1: ~13.40%</li></ul> |
| query | passage | label |
|---|---|---|
| <code>what is signifying</code> | <code>Signifyin' directs attention to the connotative, context-bound significance of words, which is accessible only to those who share the cultural values of a given speech community. While Signifyin(g) is the term coined by Henry Louis Gates, Jr. to represent a black vernacular, the idea stems from the thoughts of Ferdinand De Saussure and the process of signifying--the association between words and the ideas they indicate..</code> | <code>0</code> |
| <code>definition of scaled</code> | <code>scale noun (SIZE). B2 [S or U] the size or level of something, especially when this is large: We don't yet know the scale of the problem. Nuclear weapons cause destruction on a massive scale (= cause a lot of destruction). scale noun. › [S or U] the size or level of something, especially when this is large: the scale of sth We failed to recognize the scale of the problem. on a large/small scale. on a global/national/international scale.</code> | <code>0</code> |
| <code>what is planibel glass</code> | <code>Product Details. Planibel A is the new high performance LOW-E hard coating specially developed to reach the high ranking within the WER scale (Window Energy Rating).</code> | <code>0</code> |
{
"activation_fn": "torch.nn.modules.linear.Identity",
"pos_weight": null
}
| query | passage | label | |
|---|---|---|---|
| type | string | string | int |
| details | <ul><li>min: 12 characters</li><li>mean: 34.24 characters</li><li>max: 99 characters</li></ul> | <ul><li>min: 79 characters</li><li>mean: 421.13 characters</li><li>max: 1061 characters</li></ul> | <ul><li>0: ~88.60%</li><li>1: ~11.40%</li></ul> |
| query | passage | label |
|---|---|---|
| <code>what is a ladyfinger</code> | <code>A light, delicate sponge cake roughly shaped like a large, fat finger. Used as an accompaniment to ice cream, puddings and other desserts. One of the oldest and most delicate of sponge cakes, from the House of Savoy in eleventh century France. American ladyfingers are smaller and moister than their Italian counterparts.</code> | <code>0</code> |
| <code>what is itp blood disorder</code> | <code>Immune thrombocytopenia (THROM-bo-si-toe-PE-ne-ah), or ITP, is a bleeding disorder. In ITP, the blood doesn't clot as it should. This is due to a low number of blood cell fragments called platelets (PLATE-lets) or thrombocytes (THROM-bo-sites). Platelets are made in your bone marrow along with other kinds of blood cells.</code> | <code>0</code> |
| <code>what is spark programming</code> | <code>Spark 1.5.1 works with Java 7 and higher. If you are using Java 8, Spark supports lambda expressions for concisely writing functions, otherwise you can use the classes in the org.apache.spark.api.java.function package. To write a Spark application in Java, you need to add a dependency on Spark. The first thing a Spark program must do is to create a JavaSparkContext object, which tells Spark how to access a cluster. To create a SparkContext you first need to build a SparkConf object that contains information about your application.</code> | <code>0</code> |
{
"activation_fn": "torch.nn.modules.linear.Identity",
"pos_weight": null
}
eval_strategy: epochper_device_train_batch_size: 512per_device_eval_batch_size: 512learning_rate: 2e-08warmup_ratio: 0.1seed: 12bf16: Trueload_best_model_at_end: Trueoverwrite_output_dir: Falsedo_predict: Falseeval_strategy: epochprediction_loss_only: Trueper_device_train_batch_size: 512per_device_eval_batch_size: 512per_gpu_train_batch_size: Noneper_gpu_eval_batch_size: Nonegradient_accumulation_steps: 1eval_accumulation_steps: Nonetorch_empty_cache_steps: Nonelearning_rate: 2e-08weight_decay: 0.0adam_beta1: 0.9adam_beta2: 0.999adam_epsilon: 1e-08max_grad_norm: 1.0num_train_epochs: 3max_steps: -1lr_scheduler_type: linearlr_scheduler_kwargs: {}warmup_ratio: 0.1warmup_steps: 0log_level: passivelog_level_replica: warninglog_on_each_node: Truelogging_nan_inf_filter: Truesave_safetensors: Truesave_on_each_node: Falsesave_only_model: Falserestore_callback_states_from_checkpoint: Falseno_cuda: Falseuse_cpu: Falseuse_mps_device: Falseseed: 12data_seed: Nonejit_mode_eval: Falseuse_ipex: Falsebf16: Truefp16: Falsefp16_opt_level: O1half_precision_backend: autobf16_full_eval: Falsefp16_full_eval: Falsetf32: Nonelocal_rank: 0ddp_backend: Nonetpu_num_cores: Nonetpu_metrics_debug: Falsedebug: []dataloader_drop_last: Falsedataloader_num_workers: 0dataloader_prefetch_factor: Nonepast_index: -1disable_tqdm: Falseremove_unused_columns: Truelabel_names: Noneload_best_model_at_end: Trueignore_data_skip: Falsefsdp: []fsdp_min_num_params: 0fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}tp_size: 0fsdp_transformer_layer_cls_to_wrap: Noneaccelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}deepspeed: Nonelabel_smoothing_factor: 0.0optim: adamw_torchoptim_args: Noneadafactor: Falsegroup_by_length: Falselength_column_name: lengthddp_find_unused_parameters: Noneddp_bucket_cap_mb: Noneddp_broadcast_buffers: Falsedataloader_pin_memory: Truedataloader_persistent_workers: Falseskip_memory_metrics: Trueuse_legacy_prediction_loop: Falsepush_to_hub: Falseresume_from_checkpoint: Nonehub_model_id: Nonehub_strategy: every_savehub_private_repo: Nonehub_always_push: Falsegradient_checkpointing: Falsegradient_checkpointing_kwargs: Noneinclude_inputs_for_metrics: Falseinclude_for_metrics: []eval_do_concat_batches: Truefp16_backend: autopush_to_hub_model_id: Nonepush_to_hub_organization: Nonemp_parameters:auto_find_batch_size: Falsefull_determinism: Falsetorchdynamo: Noneray_scope: lastddp_timeout: 1800torch_compile: Falsetorch_compile_backend: Nonetorch_compile_mode: Nonedispatch_batches: Nonesplit_batches: Noneinclude_tokens_per_second: Falseinclude_num_input_tokens_seen: Falseneftune_noise_alpha: Noneoptim_target_modules: Nonebatch_eval_metrics: Falseeval_on_start: Falseuse_liger_kernel: Falseeval_use_gather_object: Falseaverage_tokens_across_devices: Falseprompts: Nonebatch_sampler: batch_samplermulti_dataset_batch_sampler: proportional| Epoch | Step | Training Loss | Validation Loss | NanoMSMARCO_R100_ndcg@10 | NanoNFCorpus_R100_ndcg@10 | NanoNQ_R100_ndcg@10 | NanoBEIR_R100_mean_ndcg@10 |
|---|---|---|---|---|---|---|---|
| -1 | -1 | - | - | 0.0281 (-0.5123) | 0.2158 (-0.1092) | 0.0350 (-0.4656) | 0.0930 (-0.3624) |
| 0.0008 | 1 | 0.6816 | - | - | - | - | - |
| 0.7680 | 1000 | 0.6802 | - | - | - | - | - |
| 1.0 | 1302 | - | 0.6756 | 0.0324 (-0.5081) | 0.2171 (-0.1079) | 0.0330 (-0.4677) | 0.0941 (-0.3612) |
| 1.5361 | 2000 | 0.6752 | - | - | - | - | - |
| 2.0 | 2604 | - | 0.6710 | 0.0507 (-0.4898) | 0.2166 (-0.1084) | 0.0184 (-0.4823) | 0.0952 (-0.3602) |
| 2.3041 | 3000 | 0.6718 | - | - | - | - | - |
| 3.0 | 3906 | - | 0.6693 | 0.0507 (-0.4898) | 0.2176 (-0.1074) | 0.0300 (-0.4706) | 0.0994 (-0.3559) |
| -1 | -1 | - | - | 0.0507 (-0.4898) | 0.2176 (-0.1074) | 0.0300 (-0.4706) | 0.0994 (-0.3559) |
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
<!--
## Glossary
*Clearly define terms in order to be accessible across audiences.*
-->
<!--
## Model Card Authors
*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->
<!--
## Model Card Contact
*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->