kd-nfs-bm25cat

CrossEncoder based on microsoft/MiniLM-L12-H384-uncased

This is a Cross Encoder model finetuned from microsoft/MiniLM-L12-H384-uncased using the sentence-transformers library. It computes scores for pairs of texts, which can be used for text reranking and semantic search.

Model Details

Model Description

• Model Type: Cross Encoder
• Base model: microsoft/MiniLM-L12-H384-uncased <!-- at revision 44acabbec0ef496f6dbc93adadea57f376b7c0ec -->
• Maximum Sequence Length: 256 tokens
• Number of Output Labels: 1 label

<!-- - **Training Dataset:** Unknown --> <!-- - **Language:** Unknown --> <!-- - **License:** Unknown -->

Model Sources

• Documentation: Sentence Transformers Documentation
• Documentation: Cross Encoder Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Cross Encoders on Hugging Face

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import CrossEncoder

# Download from the 🤗 Hub
model = CrossEncoder("cross_encoder_model_id")
# Get scores for pairs of texts
pairs = [
    ['19 [SEP] average cost of dj for wedding', "The Knot found the average cost of a wedding nationwide is $35,329 not including the honeymoon. That's up $2,688 from last year and it's the highest average since The Knot started tracking average wedding costs 10 years ago. So what's behind the jump?"],
    ['15 [SEP] what is the origin name glaser mean', 'Paul Michael Glaser. Paul Michael Glaser (born March 25, 1943) is an American actor and director, perhaps best known for his role as Detective David Starsky on the 1970s television series Starsky and Hutch; he also appeared as Captain Jack Steeper on the 1999 to 2005 NBC series Third Watch.'],
    ['14 [SEP] what is the ego', 'The ego is  your consciousness of your own identity, or the conscious mind. Basically, your ego is related to your confidence level; if you have a big ego, your confidence is high, and visa versa. Additionally, having an ego can mean having an inflated feeling of pride or self-worth. The ego is your consciousness of your own identity, or the conscious mind. Basically, your ego is related to your confidence level; if you have a big ego, your confidence is high, and visa versa. Additionally, having an ego can mean having an inflated feeling of pride or self-worth.'],
    ['10 [SEP] what is the virginia mason emmi program', "Although Oklahoma State receiver James Washington is only 6 feet tall, quarterback Mason Rudolph doesn't see his size as a negative in today's NFL. Although Oklahoma State receiver James Washington is only 6 feet tall, quarterback Mason Rudolph doesn't see his size as a negative in today's NFL."],
    ['16 [SEP] is there an airport in atlantic city?', "The Absecon station is served by New Jersey Transit 's Atlantic City Line trains, with east-west service between 30th Street Station in Philadelphia and the Atlantic City Rail Terminal in Atlantic City."],
]
scores = model.predict(pairs)
print(scores.shape)
# (5,)

# Or rank different texts based on similarity to a single text
ranks = model.rank(
    '19 [SEP] average cost of dj for wedding',
    [
        "The Knot found the average cost of a wedding nationwide is $35,329 not including the honeymoon. That's up $2,688 from last year and it's the highest average since The Knot started tracking average wedding costs 10 years ago. So what's behind the jump?",
        'Paul Michael Glaser. Paul Michael Glaser (born March 25, 1943) is an American actor and director, perhaps best known for his role as Detective David Starsky on the 1970s television series Starsky and Hutch; he also appeared as Captain Jack Steeper on the 1999 to 2005 NBC series Third Watch.',
        'The ego is  your consciousness of your own identity, or the conscious mind. Basically, your ego is related to your confidence level; if you have a big ego, your confidence is high, and visa versa. Additionally, having an ego can mean having an inflated feeling of pride or self-worth. The ego is your consciousness of your own identity, or the conscious mind. Basically, your ego is related to your confidence level; if you have a big ego, your confidence is high, and visa versa. Additionally, having an ego can mean having an inflated feeling of pride or self-worth.',
        "Although Oklahoma State receiver James Washington is only 6 feet tall, quarterback Mason Rudolph doesn't see his size as a negative in today's NFL. Although Oklahoma State receiver James Washington is only 6 feet tall, quarterback Mason Rudolph doesn't see his size as a negative in today's NFL.",
        "The Absecon station is served by New Jersey Transit 's Atlantic City Line trains, with east-west service between 30th Street Station in Philadelphia and the Atlantic City Rail Terminal in Atlantic City.",
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

<!-- ### Direct Usage (Transformers) <details><summary>Click to see the direct usage in Transformers</summary> </details> --> <!-- ### Downstream Usage (Sentence Transformers) You can finetune this model on your own dataset. <details><summary>Click to expand</summary> </details> --> <!-- ### Out-of-Scope Use *List how the model may foreseeably be misused and address what users ought not to do with the model.* --> <!-- ## Bias, Risks and Limitations *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* --> <!-- ### Recommendations *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* -->

Training Details

Training Dataset

Unnamed Dataset

• Size: 79,518,560 training samples
• Columns: <code>sentence_0</code>, <code>sentence_1</code>, and <code>label</code>
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1	label
  type	string	string	float
  details	<ul><li>min: 19 characters</li><li>mean: 42.8 characters</li><li>max: 138 characters</li></ul>	<ul><li>min: 67 characters</li><li>mean: 346.56 characters</li><li>max: 916 characters</li></ul>	<ul><li>min: -11.84</li><li>mean: 0.64</li><li>max: 11.02</li></ul>

• Samples:

  sentence_0	sentence_1	label
  <code>19 [SEP] average cost of dj for wedding</code>	<code>The Knot found the average cost of a wedding nationwide is $35,329 not including the honeymoon. That's up $2,688 from last year and it's the highest average since The Knot started tracking average wedding costs 10 years ago. So what's behind the jump?</code>	<code>0.9657705624898275</code>
  <code>15 [SEP] what is the origin name glaser mean</code>	<code>Paul Michael Glaser. Paul Michael Glaser (born March 25, 1943) is an American actor and director, perhaps best known for his role as Detective David Starsky on the 1970s television series Starsky and Hutch; he also appeared as Captain Jack Steeper on the 1999 to 2005 NBC series Third Watch.</code>	<code>1.0410912434260051</code>
  <code>14 [SEP] what is the ego</code>	<code>The ego is your consciousness of your own identity, or the conscious mind. Basically, your ego is related to your confidence level; if you have a big ego, your confidence is high, and visa versa. Additionally, having an ego can mean having an inflated feeling of pride or self-worth. The ego is your consciousness of your own identity, or the conscious mind. Basically, your ego is related to your confidence level; if you have a big ego, your confidence is high, and visa versa. Additionally, having an ego can mean having an inflated feeling of pride or self-worth.</code>	<code>8.833635489145914</code>

• Loss: <code>FitMixinLoss</code>

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 32
• per_device_eval_batch_size: 32
• num_train_epochs: 1
• fp16: True

All Hyperparameters

<details><summary>Click to expand</summary>

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 32
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

</details>

Training Logs

<details><summary>Click to expand</summary>

The model card is truncated. Read the rest on Hugging Face.