all-MiniLM-L6-v64-pair_score

all-MiniLM-L6-v64-pair_score

This is a sentence-transformers model finetuned from sentence-transformers/all-MiniLM-L6-v2 on the pairs_with_scores_v50_fixed dataset. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/all-MiniLM-L6-v2 <!-- at revision 1110a243fdf4706b3f48f1d95db1a4f5529b4d41 -->
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • pairs_with_scores_v50_fixed
• Language: en
• License: apache-2.0

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'BertModel'})
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'bathroom towel',
    'ikt wax hair stick nongreasy stick what it is stick - wax stick for hair slick stick for hair fly away amp edge frizz easy to absorb amp no white chips amp no-greasy',
    'rillettes de canard shredded duck meat duck jar duck rillettes jar shredded duck meat.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000, -0.1124, -0.0596],
#         [-0.1124,  1.0000,  0.1770],
#         [-0.0596,  0.1770,  1.0000]])

<!-- ### Direct Usage (Transformers) <details><summary>Click to see the direct usage in Transformers</summary> </details> --> <!-- ### Downstream Usage (Sentence Transformers) You can finetune this model on your own dataset. <details><summary>Click to expand</summary> </details> --> <!-- ### Out-of-Scope Use *List how the model may foreseeably be misused and address what users ought not to do with the model.* --> <!-- ## Bias, Risks and Limitations *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* --> <!-- ### Recommendations *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* -->

Training Details

Training Dataset

pairs_with_scores_v50_fixed

• Dataset: pairs_with_scores_v50_fixed at 1dd3b7b
• Size: 45,541,092 training samples
• Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	score
  type	string	string	float
  details	<ul><li>min: 4 tokens</li><li>mean: 7.37 tokens</li><li>max: 23 tokens</li></ul>	<ul><li>min: 4 tokens</li><li>mean: 60.63 tokens</li><li>max: 256 tokens</li></ul>	<ul><li>min: 0.0</li><li>mean: 0.2</li><li>max: 1.0</li></ul>

• Samples:

  sentence1	sentence2	score
  <code>banana vegan protein</code>	<code>melamine square melamine dark green. the product will melt under microwave heat. dishwasher proof.</code>	<code>0.0</code>
  <code>assorted cande</code>	<code>banana cinnamon oats pancake banana cinnamon pancake banana pancake oats cinnamon pancake</code>	<code>0.0</code>
  <code>passion fruit milk tea</code>	<code>dry large cotton postpartum pads - 10 per pack dry cotton pads</code>	<code>0.0</code>

• Loss: <code>CoSENTLoss</code> with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }
  

Evaluation Dataset

pairs_with_scores_v50_fixed

• Dataset: pairs_with_scores_v50_fixed at 1dd3b7b
• Size: 228,850 evaluation samples
• Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	score
  type	string	string	float
  details	<ul><li>min: 4 tokens</li><li>mean: 7.49 tokens</li><li>max: 30 tokens</li></ul>	<ul><li>min: 4 tokens</li><li>mean: 59.61 tokens</li><li>max: 249 tokens</li></ul>	<ul><li>min: 0.0</li><li>mean: 0.21</li><li>max: 1.0</li></ul>

• Samples:

  sentence1	sentence2	score
  <code>long sleeves jersi</code>	<code>soap dispenser 2 x 1 with cleaning sponge soap pump dispenser soap dispenser for kitchen dishwashing dispenser soap dispenser with sponge 2-in-1 soap pump dispenser sponge holder set with sponge for dish soap and kitchenfeature 1. unlike other traditional dispensers this 2-in-1 dishwasher soap dispenser is designed to be superior made of unbreakable abs plastic making it long-lasting. no more stopping to grab the soap bottle. any other sponge or scrubber will work with it too. 2. sink dispenser with attached caddy holds sponges scrubbers rags scouring pads in one convenient place while keeping water off of kitchen counters. about 5.1 x 3.35 x 3.5 inches. 3. dispense the perfect amount of soap onto the sponge extremely easy to use. the container has a large capacity of 13 ounces so that you don t need to refill it freque</code>	<code>0.0</code>
  <code>off light olive handwoven linen cardigan three quarter</code>	<code>flex tape patch bond and seal waterproof seal repair fast drying tape seal tape flex tape patch bond - super strong - rubberized waterproof seal repair - seal leaky hoses pipe tubing - works fast and dries in one hour - flex tape is a super strong rubberized waterproof tape that can patch bond seal and repair virtually everything. flex tape is specially formulated with a thick flexible rubberized backing that conforms to any shape or object. flex tape can be applied to hot or cold wet or dry ..even underwater surfaces. flex tape instantly seals out water air and moisture to create a super strong watertight barrier. flex tape comes in black. it s extra wide size allows you to quickly cover large cracks gaps and holes. flex tape uv resistant environ</code>	<code>0.0</code>
  <code>yarn pom poms</code>	<code>black plain swimsuit microfiber swimsuit printed swimsuit polyester swimsuit pockets swimsuit plain bathing costume plain bathing suit microfiber fabric plain 100 polyester fabric 2 front side pockets 1 back pocket</code>	<code>0.0</code>

• Loss: <code>CoSENTLoss</code> with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 64
• learning_rate: 2e-05
• num_train_epochs: 1
• warmup_ratio: 0.1
• fp16: True

All Hyperparameters

<details><summary>Click to expand</summary>

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 64
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

</details>

Training Logs

<details><summary>Click to expand</summary>

The model card is truncated. Read the rest on Hugging Face.