all-MiniLM-L6-v66-pair_score

all-MiniLM-L6-v66-pair_score

This is a sentence-transformers model finetuned from sentence-transformers/all-MiniLM-L6-v2 on the pairs_with_scores_v53 dataset. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/all-MiniLM-L6-v2 <!-- at revision 1110a243fdf4706b3f48f1d95db1a4f5529b4d41 -->
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • pairs_with_scores_v53
• Language: en
• License: apache-2.0

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'BertModel'})
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'colorful marble tote',
    'baygon crawling insects killer 400m baygon ant killer baygon insect killer baygon roach killer roaches killer baygon bug killer baygon crawling bugs killer',
    'linen kaftan inlaid with indian silk breathable kaftan inlaid with indian linen galebaya inlaid with indian a linen kaftan adorned with indian silk combines elegance with luxury. this kaftan features linen fabric providing comfort and breathability ideal for warmer days along with indian silk detailing that adds a touch of opulence and beauty. the indian silk embellishments may adorn the edges neckline or as elegant motifs throughout the kaftan. this garment is perfect for special occasions or times when a sophisticated and distinctive look is desired.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000, -0.0657,  0.0665],
#         [-0.0657,  1.0000, -0.0702],
#         [ 0.0665, -0.0702,  1.0000]])

<!-- ### Direct Usage (Transformers) <details><summary>Click to see the direct usage in Transformers</summary> </details> --> <!-- ### Downstream Usage (Sentence Transformers) You can finetune this model on your own dataset. <details><summary>Click to expand</summary> </details> --> <!-- ### Out-of-Scope Use *List how the model may foreseeably be misused and address what users ought not to do with the model.* --> <!-- ## Bias, Risks and Limitations *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* --> <!-- ### Recommendations *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* -->

Training Details

Training Dataset

pairs_with_scores_v53

• Dataset: pairs_with_scores_v53 at 5056e53
• Size: 12,818,275 training samples
• Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	score
  type	string	string	float
  details	<ul><li>min: 4 tokens</li><li>mean: 7.46 tokens</li><li>max: 23 tokens</li></ul>	<ul><li>min: 4 tokens</li><li>mean: 58.78 tokens</li><li>max: 256 tokens</li></ul>	<ul><li>min: 0.0</li><li>mean: 0.27</li><li>max: 1.0</li></ul>

• Samples:

  sentence1	sentence2	score
  <code>casting creme gloss hair color</code>	<code>dabur vatika burgundy henna dabur vatika henna</code>	<code>1.0</code>
  <code>wooden wick candle</code>	<code>twist drink - apple - 0.2 l apple soda twist and drink soda twist - apple twist and soda</code>	<code>0.0</code>
  <code>hugo boss pure edt</code>	<code>vision invincible edt 100 ml tonka bean eau de toilette long lasting eau de toilette invincible eau de toilette vision eau de toilette introducing vision invincible edt 100ml a luxurious fragrance that will awaken your senses and leave you feeling invincible. this exquisite scent is a medley of italian lemon ceylon cinnamon dry amber tonka bean vanilla and moss creating a luscious and irresistible aroma. indulge in the splendor of this long-lasting fragrance perfect for any occasion. the top notes of italy lemon bergamot and green apple will captivate your senses while the heart notes of dry amber tonka beans and plum add a touch of warmth and depth. as the fragrance settles the base notes of vanilla amber moss and dry wood will leave a lingering and alluring scent that will make you feel c</code>	<code>1.0</code>

• Loss: <code>CoSENTLoss</code> with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }
  

Evaluation Dataset

pairs_with_scores_v53

• Dataset: pairs_with_scores_v53 at 5056e53
• Size: 64,414 evaluation samples
• Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	score
  type	string	string	float
  details	<ul><li>min: 4 tokens</li><li>mean: 7.43 tokens</li><li>max: 22 tokens</li></ul>	<ul><li>min: 4 tokens</li><li>mean: 57.88 tokens</li><li>max: 256 tokens</li></ul>	<ul><li>min: 0.0</li><li>mean: 0.25</li><li>max: 1.0</li></ul>

• Samples:

  sentence1	sentence2	score
  <code>kaya wood dinnerware</code>	<code>vascomedra 600mg 30/f.c.tab</code>	<code>0.0</code>
  <code>mothers day bracelet round</code>	<code>circle bracelet men bracelet</code>	<code>1.0</code>
  <code>rubber sole slipper</code>	<code>switchbot curtain u rail smart home robot curtain motorized robot electrical home tool curtain automated robot electrical curtain rail home automation device switchbot curtain is a smart home robot that makes your existing curtain motorized and automated with 30 second installation schedule and touch go. u rail is required.</code>	<code>0.0</code>

• Loss: <code>CoSENTLoss</code> with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 64
• learning_rate: 2e-05
• num_train_epochs: 1
• warmup_ratio: 0.1
• fp16: True

All Hyperparameters

<details><summary>Click to expand</summary>

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 64
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

</details>

Training Logs

<details><summary>Click to expand</summary>

The model card is truncated. Read the rest on Hugging Face.