all-MiniLM-L6-v75-pair_score

all-MiniLM-L6-v75-pair_score

This is a sentence-transformers model finetuned from KhaledReda/all-MiniLM-L6-v73-pair_score on the pairs_with_scores_v59 dataset. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: KhaledReda/all-MiniLM-L6-v73-pair_score <!-- at revision 8d7290c3e3a83fe2a2f9960f8898af79520ac801 -->
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • pairs_with_scores_v59
• Language: en
• License: apache-2.0

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'BertModel'})
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'pet grenadine syrup',
    'health liceel moist.shamp.200m health liceel shampoo health shampoo',
    'el bawadi - molasses - 355 gr el bawadi molasses',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000,  0.0037,  0.0142],
#         [ 0.0037,  1.0000, -0.0912],
#         [ 0.0142, -0.0912,  1.0000]])

<!-- ### Direct Usage (Transformers) <details><summary>Click to see the direct usage in Transformers</summary> </details> --> <!-- ### Downstream Usage (Sentence Transformers) You can finetune this model on your own dataset. <details><summary>Click to expand</summary> </details> --> <!-- ### Out-of-Scope Use *List how the model may foreseeably be misused and address what users ought not to do with the model.* --> <!-- ## Bias, Risks and Limitations *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* --> <!-- ### Recommendations *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* -->

Training Details

Training Dataset

pairs_with_scores_v59

• Dataset: pairs_with_scores_v59 at 42432d1
• Size: 21,759,025 training samples
• Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	score
  type	string	string	float
  details	<ul><li>min: 4 tokens</li><li>mean: 7.39 tokens</li><li>max: 22 tokens</li></ul>	<ul><li>min: 4 tokens</li><li>mean: 59.02 tokens</li><li>max: 256 tokens</li></ul>	<ul><li>min: 0.0</li><li>mean: 0.01</li><li>max: 1.0</li></ul>

• Samples:

  sentence1	sentence2	score
  <code>rust baking cones</code>	<code>oral-b satin tape dental mint dental floss oralb satin tape</code>	<code>0.0</code>
  <code>moon cake cookie cutter</code>	<code>neutrogena bright boost spf 30 fluid gel 50 ml neutrogena fluid gel</code>	<code>0.0</code>
  <code>mini pavlova</code>	<code>travel set body mist longlasting body mist lightweight body mist set body splash set travel set body splash travel set indulge your senses with our body mist. this luxurious mist is specially formulated to enhance your overall routine by providing a refreshing and long-lasting fragrance for your body. crafted with care our mist will envelop you in a captivating scent leaving you feeling confident and invigorated throughout the day. the lightweight formula is designed to be easily absorbed leaving no residue and keeping your skin hydrated. simply spritz it on for a rejuvenating burst of fragrance that will awaken your senses and leave you feeling irresistibly fresh.</code>	<code>0.0</code>

• Loss: <code>CoSENTLoss</code> with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }
  

Evaluation Dataset

pairs_with_scores_v59

• Dataset: pairs_with_scores_v59 at 42432d1
• Size: 109,342 evaluation samples
• Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	score
  type	string	string	float
  details	<ul><li>min: 4 tokens</li><li>mean: 7.45 tokens</li><li>max: 26 tokens</li></ul>	<ul><li>min: 4 tokens</li><li>mean: 55.99 tokens</li><li>max: 256 tokens</li></ul>	<ul><li>min: 0.0</li><li>mean: 0.02</li><li>max: 1.0</li></ul>

• Samples:

  sentence1	sentence2	score
  <code>canpol blbottle silicon 59 - 300</code>	<code>pvc straps wedge heeled slipper straw slipper leather slipper rubber sole slipper open toecap slipper slip on slipper polyvinyl chloride slipper pvc slipper straps slipper wedge slipper pvc straw upper material. leather inner material. hard rubber sole. open toecap shape. slip on.</code>	<code>0.0</code>
  <code>short sleeves sleepshirt</code>	<code>shin instep guard woven polyester adidas padded boxing shin instep guard is made from 100 woven polyester with safety break away. adidas is padded for maximum safety and provided with foot protection.</code>	<code>0.0</code>
  <code>avocado matcha tart</code>	<code>satin rose hair-tie transform your hairstyle with our satin rose hair-tie. add a touch of elegance and sophistication to your look while securing your hair with this beautiful rose . elevate your everyday style and experience the luxurious feel of satin. perfect for any occasion. modeled picturea are for reference the product pictures are the exact design of the item</code>	<code>0.0</code>

• Loss: <code>CoSENTLoss</code> with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 64
• learning_rate: 2e-05
• num_train_epochs: 1
• warmup_ratio: 0.1
• fp16: True

All Hyperparameters

<details><summary>Click to expand</summary>

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 64
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

</details>

Training Logs

<details><summary>Click to expand</summary>

The model card is truncated. Read the rest on Hugging Face.