instacart-two-tower-sbert

SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2

This is a sentence-transformers model finetuned from sentence-transformers/all-MiniLM-L6-v2. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/all-MiniLM-L6-v2 <!-- at revision c9745ed1d9f207416be6d2e6f8de32d1f16199bf -->
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity

<!-- - **Training Dataset:** Unknown --> <!-- - **Language:** Unknown --> <!-- - **License:** Unknown -->

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'BertModel'})
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    '[+30d w5h19] Banana, Ezekiel 4:9 Bread Organic Sprouted Whole Grain, Grade A Cage Free Omega-3 Enriched Medium Eggs, Sweet Potatoes; [+10d w1h18] Cookie Dough Ice Cream Chocolate Chip, Peanut Butter Ice Cream Cup, Blackberry Cucumber Sparkling Water, Kiwi Sandia Sparkling Water, Protein Power Cold Pressed Fruit Smoothie, Defense Up Cold Pressed Fruit Smoothie; [+30d w4h12] Blackberry Cucumber Sparkling Water, Peanut Butter Ice Cream Cup, Cookie Dough Ice Cream Chocolate Chip, Cran Raspberry Sparkling Water, Lemon Italian Sparkling Mineral Water, Natural Chicken & Sage Breakfast Sausage, Organic Jasmine Rice, Organic Riced Cauliflower, Country Cheddar Bowl; [+5d w2h10] Original Rotisserie Chicken. Next: +16d w0h14',
    'Product: Large Lemon. Aisle: fresh fruits. Department: produce.',
    'Product: Blood Orange Italian Soda. Aisle: soft drinks. Department: beverages.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.3506, 0.3826],
#         [0.3506, 1.0000, 0.6395],
#         [0.3826, 0.6395, 1.0000]])

<!-- ### Direct Usage (Transformers) <details><summary>Click to see the direct usage in Transformers</summary> </details> --> <!-- ### Downstream Usage (Sentence Transformers) You can finetune this model on your own dataset. <details><summary>Click to expand</summary> </details> --> <!-- ### Out-of-Scope Use *List how the model may foreseeably be misused and address what users ought not to do with the model.* -->

Evaluation

Metrics

Information Retrieval

• Dataset: order-recommendation
• Evaluated with <code>InformationRetrievalEvaluator</code>

<!-- ## Bias, Risks and Limitations *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* --> <!-- ### Recommendations *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* -->

Training Details

Training Dataset

Unnamed Dataset

• Size: 1,246,220 training samples
• Columns: <code>anchor</code> and <code>positive</code>
• Approximate statistics based on the first 1000 samples:

  	anchor	positive
  type	string	string
  details	<ul><li>min: 76 tokens</li><li>mean: 151.76 tokens</li><li>max: 208 tokens</li></ul>	<ul><li>min: 15 tokens</li><li>mean: 20.12 tokens</li><li>max: 38 tokens</li></ul>

• Samples:

  anchor	positive
  <code>[w1h14] Tuna Ventresca, in Olive Oil, Lightly Smoked Sardines in Olive Oil, Naturally Smoked Oysters in Pure Olive Oil, Bulgarian Yogurt, Organic Whole String Cheese, Organic 4% Milk Fat Whole Milk Cottage Cheese, Pure Sparkling Water, Organic Small Bunch Celery; [+7d w1h10] Bulgarian Yogurt, Organic 4% Milk Fat Whole Milk Cottage Cheese, Pure Sparkling Water, Organic Small Bunch Celery, Organic Whole String Cheese, Banana, Plus Cranberry Almond + Antioxidants with Macadamia Nuts Bar, Dark Chocolate Cinnamon Pecan Bar; [+15d w2h21] Pure Sparkling Water, Dark Chocolate Cinnamon Pecan Bar, Sparkling Water Grapefruit, Naturally Smoked Oysters in Pure Olive Oil. Next: +9d w4h10</code>	<code>Product: Bulgarian Yogurt. Aisle: yogurt. Department: dairy eggs.</code>
  <code>[w1h14] Tuna Ventresca, in Olive Oil, Lightly Smoked Sardines in Olive Oil, Naturally Smoked Oysters in Pure Olive Oil, Bulgarian Yogurt, Organic Whole String Cheese, Organic 4% Milk Fat Whole Milk Cottage Cheese, Pure Sparkling Water, Organic Small Bunch Celery; [+7d w1h10] Bulgarian Yogurt, Organic 4% Milk Fat Whole Milk Cottage Cheese, Pure Sparkling Water, Organic Small Bunch Celery, Organic Whole String Cheese, Banana, Plus Cranberry Almond + Antioxidants with Macadamia Nuts Bar, Dark Chocolate Cinnamon Pecan Bar; [+15d w2h21] Pure Sparkling Water, Dark Chocolate Cinnamon Pecan Bar, Sparkling Water Grapefruit, Naturally Smoked Oysters in Pure Olive Oil. Next: +9d w4h10</code>	<code>Product: Organic 4% Milk Fat Whole Milk Cottage Cheese. Aisle: other creams cheeses. Department: dairy eggs.</code>
  <code>[w1h14] Tuna Ventresca, in Olive Oil, Lightly Smoked Sardines in Olive Oil, Naturally Smoked Oysters in Pure Olive Oil, Bulgarian Yogurt, Organic Whole String Cheese, Organic 4% Milk Fat Whole Milk Cottage Cheese, Pure Sparkling Water, Organic Small Bunch Celery; [+7d w1h10] Bulgarian Yogurt, Organic 4% Milk Fat Whole Milk Cottage Cheese, Pure Sparkling Water, Organic Small Bunch Celery, Organic Whole String Cheese, Banana, Plus Cranberry Almond + Antioxidants with Macadamia Nuts Bar, Dark Chocolate Cinnamon Pecan Bar; [+15d w2h21] Pure Sparkling Water, Dark Chocolate Cinnamon Pecan Bar, Sparkling Water Grapefruit, Naturally Smoked Oysters in Pure Olive Oil. Next: +9d w4h10</code>	<code>Product: Organic Celery Hearts. Aisle: fresh vegetables. Department: produce.</code>

• Loss: <code>MultipleNegativesRankingLoss</code> with these parameters:

  {
      "scale": 30.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Evaluation Dataset

Unnamed Dataset

• Size: 138,397 evaluation samples
• Columns: <code>anchor</code> and <code>positive</code>
• Approximate statistics based on the first 1000 samples:

  	anchor	positive
  type	string	string
  details	<ul><li>min: 66 tokens</li><li>mean: 150.77 tokens</li><li>max: 203 tokens</li></ul>	<ul><li>min: 14 tokens</li><li>mean: 20.47 tokens</li><li>max: 44 tokens</li></ul>

• Samples:

  anchor	positive
  <code>[w1h21] Large Alfresco Eggs, Organic Whole Milk, Organic Cream Cheese Bar, Organic Grape Tomatoes, Organic Raspberries, Organic Strawberries, Organic D'Anjou Pears, Organic Green Seedless Grapes, Small Hass Avocado, Organic Unsalted Diced Tomatoes, Pasta Sauce, Aged White Cheddar Gluten-Free Baked Rice And Corn Puffs, Crescent Rolls, Plain Greek Yogurt, Natural Free & Clear Dish Liquid, Organic Honey Sweet Whole Wheat Bread, Flaky Biscuits; [+13d w0h15] Strawberry Preserves, Organic Raspberries, Organic Frozen Peas. Next: +24d w0h21</code>	<code>Product: Organic Grape Tomatoes. Aisle: packaged vegetables fruits. Department: produce.</code>
  <code>[w1h21] Large Alfresco Eggs, Organic Whole Milk, Organic Cream Cheese Bar, Organic Grape Tomatoes, Organic Raspberries, Organic Strawberries, Organic D'Anjou Pears, Organic Green Seedless Grapes, Small Hass Avocado, Organic Unsalted Diced Tomatoes, Pasta Sauce, Aged White Cheddar Gluten-Free Baked Rice And Corn Puffs, Crescent Rolls, Plain Greek Yogurt, Natural Free & Clear Dish Liquid, Organic Honey Sweet Whole Wheat Bread, Flaky Biscuits; [+13d w0h15] Strawberry Preserves, Organic Raspberries, Organic Frozen Peas. Next: +24d w0h21</code>	<code>Product: Organic Strawberries. Aisle: fresh fruits. Department: produce.</code>
  <code>[w1h21] Large Alfresco Eggs, Organic Whole Milk, Organic Cream Cheese Bar, Organic Grape Tomatoes, Organic Raspberries, Organic Strawberries, Organic D'Anjou Pears, Organic Green Seedless Grapes, Small Hass Avocado, Organic Unsalted Diced Tomatoes, Pasta Sauce, Aged White Cheddar Gluten-Free Baked Rice And Corn Puffs, Crescent Rolls, Plain Greek Yogurt, Natural Free & Clear Dish Liquid, Organic Honey Sweet Whole Wheat Bread, Flaky Biscuits; [+13d w0h15] Strawberry Preserves, Organic Raspberries, Organic Frozen Peas. Next: +24d w0h21</code>	<code>Product: Organic Honey Sweet Whole Wheat Bread. Aisle: bread. Department: bakery.</code>

• Loss: <code>MultipleNegativesRankingLoss</code> with these parameters:

  {
      "scale": 30.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: epoch
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 64
• learning_rate: 0.0001
• num_train_epochs: 5
• lr_scheduler_type: cosine
• warmup_steps: 9736
• dataloader_drop_last: True
• load_best_model_at_end: True
• dataloader_pin_memory: False
• gradient_checkpointing: True
• batch_sampler: no_duplicates

All Hyperparameters

<details><summary>Click to expand</summary>

• do_predict: False
• eval_strategy: epoch
• prediction_loss_only: True
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 64
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 0.0001
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 5
• max_steps: -1
• lr_scheduler_type: cosine
• lr_scheduler_kwargs: None
• warmup_ratio: None
• warmup_steps: 9736
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• enable_jit_checkpoint: False
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• use_cpu: False
• seed: 42
• data_seed: None
• bf16: False
• fp16: False
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: -1
• ddp_backend: None
• debug: []
• dataloader_drop_last: True
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: False
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: True
• gradient_checkpointing_kwargs: None
• include_for_metrics: []
• eval_do_concat_batches: True
• auto_find_batch_size: False
• full_determinism: False
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• use_cache: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

</details>

Training Logs

<details><summary>Click to expand</summary>

The model card is truncated. Read the rest on Hugging Face.