reranker-msmarco-v1.1-ettin-encoder-68m-listnet

CrossEncoder based on jhu-clsp/ettin-encoder-68m

This is a Cross Encoder model finetuned from jhu-clsp/ettin-encoder-68m on the ms_marco dataset using the sentence-transformers library. It computes scores for pairs of texts, which can be used for text reranking and semantic search.

Model Details

Model Description

• Model Type: Cross Encoder
• Base model: jhu-clsp/ettin-encoder-68m <!-- at revision ac19ae4bc51093b31c475665ac872a936d056cc2 -->
• Maximum Sequence Length: 7999 tokens
• Number of Output Labels: 1 label
• Training Dataset:
  • ms_marco
• Language: en

<!-- - **License:** Unknown -->

Model Sources

• Documentation: Sentence Transformers Documentation
• Documentation: Cross Encoder Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Cross Encoders on Hugging Face

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import CrossEncoder

# Download from the 🤗 Hub
model = CrossEncoder("bansalaman18/reranker-msmarco-v1.1-ettin-encoder-68m-listnet")
# Get scores for pairs of texts
pairs = [
    ['what is the name of the vaccine for MMR', 'Measles, mumps and rubella (MMR) vaccines. The MMR vaccine also comes in combination with chickenpox (MMRV) for 18 month old children and contains small amounts of each of the viruses at a reduced strength and a small amount of the antibiotic neomycin. MMR is given at four years of age to children who did not get their second MMR vaccine at 18 months of age. The four year old MMR dose ends in December 2015. All people born during or since 1966 should check their immunisation status to ensure they have had two doses of a measles containing vaccine.'],
    ['what is the name of the vaccine for MMR', 'The MMR vaccine protects children against all three diseases and is given at 12 months of age. A second dose using MMRV vaccine is given at 18 months of age to also protect children from chickenpox. MMR is given at four years of age to children who did not get their second MMR vaccine at 18 months of age. The four year old MMR dose ends in December 2015. All people born during or since 1966 should check their immunisation status to ensure they have had two doses of a measles containing vaccine.'],
    ['what is the name of the vaccine for MMR', 'Measles vaccine is a vaccine that is very effective at preventing measles. After one dose 85% of children nine months of age and 95% over twelve months of age are immune. Nearly all of those who do not develop immunity after a single dose develop it after a second dose. Measles mumps rubella vaccine (MMR-II); MMR vaccine is a live attenuated viral vaccine used to induce immunity against measles, mumps and rubella.'],
    ['what is the name of the vaccine for MMR', 'The name of the measles vaccination is MMR. This is a three-in-one vaccination to protect against measles, mumps and rubella. The measles vaccine is a living, weakened form of the natural  measles virus. To make certain vaccines, viruses are weakened by  a process called cell culture adaptatio … n.. Cell culture  adaptation changes the natural measles virus so that it behaves  differently once it enters the body.'],
    ['what is the name of the vaccine for MMR', 'When the first dose of measles, mumps, rubella, and varicella vaccines is administered at ages 48 months and older, use of MMRV vaccine generally is preferred over separate injections of MMR and varicella vaccines. However, the overall risk of febrile seizures is very low for both options (about 8 out of every 10,000 children vaccinated with MMRV vaccine when they are 12-23 months old, and about 4 out of every 10,000 children vaccinated with the MMR and varicella vaccines at the same visit when they are 12-23 months old).'],
]
scores = model.predict(pairs)
print(scores.shape)
# (5,)

# Or rank different texts based on similarity to a single text
ranks = model.rank(
    'what is the name of the vaccine for MMR',
    [
        'Measles, mumps and rubella (MMR) vaccines. The MMR vaccine also comes in combination with chickenpox (MMRV) for 18 month old children and contains small amounts of each of the viruses at a reduced strength and a small amount of the antibiotic neomycin. MMR is given at four years of age to children who did not get their second MMR vaccine at 18 months of age. The four year old MMR dose ends in December 2015. All people born during or since 1966 should check their immunisation status to ensure they have had two doses of a measles containing vaccine.',
        'The MMR vaccine protects children against all three diseases and is given at 12 months of age. A second dose using MMRV vaccine is given at 18 months of age to also protect children from chickenpox. MMR is given at four years of age to children who did not get their second MMR vaccine at 18 months of age. The four year old MMR dose ends in December 2015. All people born during or since 1966 should check their immunisation status to ensure they have had two doses of a measles containing vaccine.',
        'Measles vaccine is a vaccine that is very effective at preventing measles. After one dose 85% of children nine months of age and 95% over twelve months of age are immune. Nearly all of those who do not develop immunity after a single dose develop it after a second dose. Measles mumps rubella vaccine (MMR-II); MMR vaccine is a live attenuated viral vaccine used to induce immunity against measles, mumps and rubella.',
        'The name of the measles vaccination is MMR. This is a three-in-one vaccination to protect against measles, mumps and rubella. The measles vaccine is a living, weakened form of the natural  measles virus. To make certain vaccines, viruses are weakened by  a process called cell culture adaptatio … n.. Cell culture  adaptation changes the natural measles virus so that it behaves  differently once it enters the body.',
        'When the first dose of measles, mumps, rubella, and varicella vaccines is administered at ages 48 months and older, use of MMRV vaccine generally is preferred over separate injections of MMR and varicella vaccines. However, the overall risk of febrile seizures is very low for both options (about 8 out of every 10,000 children vaccinated with MMRV vaccine when they are 12-23 months old, and about 4 out of every 10,000 children vaccinated with the MMR and varicella vaccines at the same visit when they are 12-23 months old).',
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

<!-- ### Direct Usage (Transformers) <details><summary>Click to see the direct usage in Transformers</summary> </details> --> <!-- ### Downstream Usage (Sentence Transformers) You can finetune this model on your own dataset. <details><summary>Click to expand</summary> </details> --> <!-- ### Out-of-Scope Use *List how the model may foreseeably be misused and address what users ought not to do with the model.* --> <!-- ## Bias, Risks and Limitations *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* --> <!-- ### Recommendations *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* -->

Training Details

Training Dataset

ms_marco

• Dataset: ms_marco at a47ee7a
• Size: 78,704 training samples
• Columns: <code>query</code>, <code>docs</code>, and <code>labels</code>
• Approximate statistics based on the first 1000 samples:

  	query	docs	labels
  type	string	list	list
  details	<ul><li>min: 11 characters</li><li>mean: 34.2 characters</li><li>max: 99 characters</li></ul>	<ul><li>min: 3 elements</li><li>mean: 6.50 elements</li><li>max: 10 elements</li></ul>	<ul><li>min: 3 elements</li><li>mean: 6.50 elements</li><li>max: 10 elements</li></ul>

• Samples:

  query	docs	labels
  <code>is leukemia genetic</code>	<code>["1 Certain genetic disorders, such as Down syndrome, are associated with an increased risk of leukemia. 2 Exposure to certain chemicals. 3 Exposure to certain chemicals, such as benzene — which is found in gasoline and is used by the chemical industry — also is linked to an increased risk of some kinds of leukemia. 4 Smoking. 1 Previous cancer treatment. 2 People who've had certain types of chemotherapy and radiation therapy for other cancers have an increased risk of developing certain types of leukemia. 3 Genetic disorders. 4 Genetic abnormalities seem to play a role in the development", 'Causes of Leukemia. Although researchers have studied the many cellular changes associated with leukemia, it is unknown why these changes occur. It is likely that certain risk factors are involved. Many factors (e.g., age, genetics) are unmodifiable (beyond control). It is now known that all cancers, including leukemia, begin as a mutation in the genetic material—the DNA (deoxyribonucleic aci...</code>	<code>[1, 0, 0, 0, 0, ...]</code>
  <code>what does stratified seed mean</code>	<code>['Definition: Stratification is a means of simulating the chilling and warming that seeds would endure if left outdoors in their native climate, for the winter. Some seeds will stay dormant until triggered by a certain amount of time in cold temperature or warm, damp conditions.', 'Seeds of different species are stratified at different temperatures and for different periods of time. Some need periods of cold temperatures, between 34 and 41 degrees Fahrenheit, while some need warm stratification at 68 to 84 degrees. The ginkgo is a deciduous gymnosperm, more closely related to the pine than an oak tree. The ginkgo or maidenhair tree (Ginkgo biloba) is a living fossil. The earliest known fossil leaves of the ginkgo are 270 million years old.', 'In the wild, seed dormancy is usually overcome by the seed spending time in the ground through a winter period and having its hard seed coat softened up by frost and weathering action. By doing so the seed is undergoing a natural form of stratific...</code>	<code>[1, 0, 0, 0, 0, ...]</code>
  <code>what is a calprotectin stool test tell</code>	<code>['Calprotectin is a stool (fecal) test that is used to detect inflammation in the intestines. Intestinal inflammation is associated with, for example, some bacterial infections and, in people with inflammatory bowel disease (IBD) , it is associated with disease activity and severity. The test may be ordered along with other stool tests, such as a stool culture to detect a bacterial infection, a stool white blood cell test, and/or a fecal occult blood test (FOBT) .', 'Faecal calprotectin is a substance that is released into the intestines in excess when there is any inflammation there. Its presence can mean a person has an inflammatory bowel disease such as Crohn’s disease or ulcerative colitis. These conditions can cause very similar symptoms to irritable bowel syndrome. ', "Faecal calprotectin is a biochemical measurement of calprotectin in the stool. Elevated faecal calprotectin indicates the migration of neutrophils to the intestinal mucosa, which occurs during intestinal inflammati...</code>	<code>[1, 0, 0, 0, 0, ...]</code>

• Loss: <code>ListNetLoss</code> with these parameters:

  {
      "activation_fn": "torch.nn.modules.linear.Identity",
      "mini_batch_size": 16
  }
  

Evaluation Dataset

ms_marco

• Dataset: ms_marco at a47ee7a
• Size: 1,000 evaluation samples
• Columns: <code>query</code>, <code>docs</code>, and <code>labels</code>
• Approximate statistics based on the first 1000 samples:

  	query	docs	labels
  type	string	list	list
  details	<ul><li>min: 9 characters</li><li>mean: 33.96 characters</li><li>max: 110 characters</li></ul>	<ul><li>min: 3 elements</li><li>mean: 6.50 elements</li><li>max: 10 elements</li></ul>	<ul><li>min: 3 elements</li><li>mean: 6.50 elements</li><li>max: 10 elements</li></ul>

• Samples:

  query	docs	labels
  <code>what is the name of the vaccine for MMR</code>	<code>['Measles, mumps and rubella (MMR) vaccines. The MMR vaccine also comes in combination with chickenpox (MMRV) for 18 month old children and contains small amounts of each of the viruses at a reduced strength and a small amount of the antibiotic neomycin. MMR is given at four years of age to children who did not get their second MMR vaccine at 18 months of age. The four year old MMR dose ends in December 2015. All people born during or since 1966 should check their immunisation status to ensure they have had two doses of a measles containing vaccine.', 'The MMR vaccine protects children against all three diseases and is given at 12 months of age. A second dose using MMRV vaccine is given at 18 months of age to also protect children from chickenpox. MMR is given at four years of age to children who did not get their second MMR vaccine at 18 months of age. The four year old MMR dose ends in December 2015. All people born during or since 1966 should check their immunisation status to ensur...</code>	<code>[1, 0, 0, 0, 0, ...]</code>
  <code>what is the smallest shark in the world</code>	<code>['Tweet. The smallest shark in the world is the Dwarf Lanternshark; a dogfish that lives in the Caribbean Sea. They grow to a maximum of 17 centimeters and eat small fish and shrimp. The next smallest shark in the world is the Pygmy Shark that grows up to 25 centimeters long. These sharks are very closely related.', 'The dwarf lanternshark (Etmopterus perryi) is a little-known species of dogfish shark in the family Etmopteridae and possibly the smallest shark in the world, reaching a maximum known length of 21.2 cm (8.3 in). Perhaps the smallest living shark species, male dwarf lanternsharks mature at a length of 16–17.5 cm (6.3–6.9 in) and females from a length of 15.5 cm (6.1 in) with 19–20 cm (7.5–7.9 in) long pregnant individuals known.', 'The Pale Catshark at 8.27 is the smallest shark in the world. The Pale Catshark is a rare cat shark of the Scyliorhinidae family and only specimen found on the Makassar Strait. ', "The world's biggest school of sharks was observed in Galapagos, i...</code>	<code>[1, 0, 0, 0, 0, ...]</code>
  <code>how much does nicu cost per day</code>	<code>['According to a 2010 article in Managed Care Magazine, the average cost for infants in NICU is around $3,000 per day. This average does not include surgeries or helicopter transports.', '1 The website Ncbi.nlm.nih.gov provides information regarding the NICU; the cost according to them can reach $1,800 to $2,500 per night of care. 2 ASPE.gov is another site that provides information regarding NICU costs.', "My son was in the NICU for 11 days and the cost was about $80,000--if I remember right, that also includes the cost of the c-section and my hospital stay. He was not in an incubator, and I don't know how much more the charge for an incubator would be--probably minimal.", "MrsDeLaVara:Our LO 'cost' about $350,000 for 62 days in the hospital. She was in the high level care facility for about half of the time, which cost more. That includes my delivery too since it was at the same hospital. Her NICU doctors billed $1500 a day for the high level and about $700 for the step down nursery...</code>	<code>[1, 0, 0, 0, 0, ...]</code>

• Loss: <code>ListNetLoss</code> with these parameters:

  {
      "activation_fn": "torch.nn.modules.linear.Identity",
      "mini_batch_size": 16
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• learning_rate: 2e-05
• num_train_epochs: 5
• seed: 12
• bf16: True
• load_best_model_at_end: True

All Hyperparameters

<details><summary>Click to expand</summary>

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 5
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 12
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• tp_size: 0
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

</details>

Training Logs

<details><summary>Click to expand</summary>

• The bold row denotes the saved checkpoint.

</details>

Framework Versions

• Python: 3.11.13
• Sentence Transformers: 5.0.0
• Transformers: 4.51.0
• PyTorch: 2.9.1+cu126
• Accelerate: 1.8.1
• Datasets: 3.6.0
• Tokenizers: 0.21.4-dev.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

ListNetLoss

@inproceedings{cao2007learning,
    title={Learning to Rank: From Pairwise Approach to Listwise Approach},
    author={Cao, Zhe and Qin, Tao and Liu, Tie-Yan and Tsai, Ming-Feng and Li, Hang},
    booktitle={Proceedings of the 24th international conference on Machine learning},
    pages={129--136},
    year={2007}
}

<!-- ## Glossary *Clearly define terms in order to be accessible across audiences.* --> <!-- ## Model Card Authors *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.* --> <!-- ## Model Card Contact *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.* -->