AIE5-MidTerm-finetuned-embeddings

SentenceTransformer based on Snowflake/snowflake-arctic-embed-l

This is a sentence-transformers model finetuned from Snowflake/snowflake-arctic-embed-l. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: Snowflake/snowflake-arctic-embed-l <!-- at revision d8fb21ca8d905d2832ee8b96c894d3298964346b -->
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 1024 dimensions
• Similarity Function: Cosine Similarity

<!-- - **Training Dataset:** Unknown --> <!-- - **Language:** Unknown --> <!-- - **License:** Unknown -->

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("thomfoolery/AIE5-MidTerm-finetuned-embeddings")
# Run inference
sentences = [
    'How can you access step and property information on the custom configuration page?',
    "Implementing a custom configuration page\n\nTo build a custom configuration page, you'll use Shopify App Bridge to render a page from your app page in Shopify Flow.\n\nFrom the context of the custom configuration page, you can then access step and property information that you can use to display the appropriate information.\n\nYou can also add additional buttons to the App Bridge title bar, or trigger a redirect to the previous page.\n\nUse Shopify App Bridge to render your app page\n\nNote: The specifics of the Custom Configuration Page integration varies between Shopify App Bridge versions. Make sure you implement the integration specific to your Shopify App Bridge version.\n\nTo render your custom configuration page, you need to integrate Shopify App Bridge on the route that you want to render. To learn about setting up Shopify App Bridge, refer to one of the following pages:\n\nGetting started with Shopify App Bridge\n\nGetting started with App Bridge React\n\nAccess action information",
    "To give merchants a more seamless action configuration experience, and to allow them to manage resources that are external to Shopify Flow, you can embed a page from your app in the Shopify Flow editor.\n\nIn your Shopify Flow action configuration, merchants see a preview with an image and text that's fetched from your custom configuration page preview URL. Merchants can click the button to access the custom configuration page.\n\nYour custom configuration page is then displayed in a frame in the Shopify admin.\n\nIn this tutorial, you'll learn how to render a custom configuration page in Shopify Flow, customize the page frame, and access data relevant to your action in the custom configuration page context.\n\nRequirements\n\nYou've created a Partner account.\n\nYou've created an app.\n\nResources\n\nTo implement this feature, you'll use the following:\n\nShopify App Bridge\n\nApp Bridge components\n\nApp Bridge actions specific to the custom configuration page\n\nImplementing a custom configuration page",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

<!-- ### Direct Usage (Transformers) <details><summary>Click to see the direct usage in Transformers</summary> </details> --> <!-- ### Downstream Usage (Sentence Transformers) You can finetune this model on your own dataset. <details><summary>Click to expand</summary> </details> --> <!-- ### Out-of-Scope Use *List how the model may foreseeably be misused and address what users ought not to do with the model.* -->

Evaluation

Metrics

Information Retrieval

• Evaluated with <code>InformationRetrievalEvaluator</code>

<!-- ## Bias, Risks and Limitations *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* --> <!-- ### Recommendations *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* -->

Training Details

Training Dataset

Unnamed Dataset

• Size: 164 training samples
• Columns: <code>sentence_0</code> and <code>sentence_1</code>
• Approximate statistics based on the first 164 samples:

  	sentence_0	sentence_1
  type	string	string
  details	<ul><li>min: 13 tokens</li><li>mean: 19.51 tokens</li><li>max: 32 tokens</li></ul>	<ul><li>min: 77 tokens</li><li>mean: 198.13 tokens</li><li>max: 309 tokens</li></ul>

• Samples:

  sentence_0	sentence_1
  <code>What is the purpose of templates in Shopify Flow?</code>	<code>How templates work<br><br>A template in Shopify Flow is an example workflow that can be copied into a merchant's shop. Templates help merchants automate a specific use case with minimal or no additional configuration. Flow's template library offers hundreds of templates with features to filter, browse, and search. You can create a template for Shopify Flow that showcases your custom triggers and actions and help merchants do more.<br><br>Next steps<br><br>Follow our step by step guide on creating a Flow template.<br><br>Learn more about how localize your template, the approval process, and more in the reference resource</code>
  <code>How can merchants utilize the template library in Shopify Flow?</code>	<code>How templates work<br><br>A template in Shopify Flow is an example workflow that can be copied into a merchant's shop. Templates help merchants automate a specific use case with minimal or no additional configuration. Flow's template library offers hundreds of templates with features to filter, browse, and search. You can create a template for Shopify Flow that showcases your custom triggers and actions and help merchants do more.<br><br>Next steps<br><br>Follow our step by step guide on creating a Flow template.<br><br>Learn more about how localize your template, the approval process, and more in the reference resource</code>
  <code>What is required for the test web server to successfully send information to Shopify Flow?</code>	<code>Requirements<br><br>Make sure that you have the following:<br><br>A test web server that you can use to send information to Shopify Flow. You can use an existing web server. This web server needs to be able to send POST requests to Shopify's GraphQL Admin API.<br><br>A test app that works with the test web server and can send HTTP requests.<br><br>A development store that has Shopify Flow and the test app installed.<br><br>Your application has access to the read_customers scope. The trigger you will build using this tutorial will be using a customer reference which requires that scope.<br><br>Step 1: Create a Flow trigger extension<br><br>To give your Flow action a meaningful name, use the following guidelines:<br><br>Use an object acted on + past tense verb format. For example, Auction bid placed.<br><br>Use sentence case.<br><br>Don't use punctuation.<br><br>Separate words using spaces.<br><br>Using Shopify CLI<br><br>The following steps show how to create a trigger that sends bid information to Shopify Flow when a bid is placed on an auction.</code>

• Loss: <code>MatryoshkaLoss</code> with these parameters:

  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          768,
          512,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 10
• per_device_eval_batch_size: 10
• num_train_epochs: 10
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

<details><summary>Click to expand</summary>

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 10
• per_device_eval_batch_size: 10
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 10
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin

</details>

Training Logs

Framework Versions

• Python: 3.11.11
• Sentence Transformers: 3.4.1
• Transformers: 4.48.3
• PyTorch: 2.5.1+cu124
• Accelerate: 1.3.0
• Datasets: 3.3.2
• Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

<!-- ## Glossary *Clearly define terms in order to be accessible across audiences.* --> <!-- ## Model Card Authors *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.* --> <!-- ## Model Card Contact *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.* -->