snoweu

SentenceTransformer based on Snowflake/snowflake-arctic-embed-m-v1.5

This is a sentence-transformers model finetuned from Snowflake/snowflake-arctic-embed-m-v1.5. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: Snowflake/snowflake-arctic-embed-m-v1.5 <!-- at revision 8e4eaca09c27ad3d501908636ec7c8bc3561b6de -->
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

<!-- - **Training Dataset:** Unknown --> <!-- - **Language:** Unknown --> <!-- - **License:** Unknown -->

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'What types of substances or mixtures should be listed in relation to their potential to react and create hazardous situations, and what additional information is required to manage the associated risks?',
    'Families of substances or mixtures or specific substances, such as water, air, acids, bases, oxidising agents, with which the substance or mixture could react to produce a hazardous situation (like an explosion, a release of toxic or flammable materials, or a liberation of excessive heat), shall be listed and if appropriate a brief description of measures to be taken to manage risks associated with such hazards shall be given.\n\n10.6. Hazardous decomposition products\n\nKnown and reasonably anticipated hazardous decomposition products produced as a result of use, storage, spill and heating shall be listed. Hazardous combustion products shall be included in section 5 of the safety data sheet.\n\n11. SECTION 11: Toxicological information',
    'The undertaking shall specify as part of the contextual information, whether the targets that it has set and presented are mandatory (required by legislation) or voluntary.\n\nDisclosure Requirement E2-4 – Pollution of air, water and soil\n\nThe undertaking shall disclose the pollutants that it emits through its own operations, as well as the microplastics it generates or uses.\n\nThe objective of this Disclosure Requirement is to provide an understanding of the emissions that the undertaking generates to air, water and soil in its own operations, and of its generation and use of microplastics.\n\nThe undertaking shall disclose the amounts of:\n\n(a)',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

<!-- ### Direct Usage (Transformers) <details><summary>Click to see the direct usage in Transformers</summary> </details> --> <!-- ### Downstream Usage (Sentence Transformers) You can finetune this model on your own dataset. <details><summary>Click to expand</summary> </details> --> <!-- ### Out-of-Scope Use *List how the model may foreseeably be misused and address what users ought not to do with the model.* -->

Evaluation

Metrics

Information Retrieval

• Evaluated with <code>InformationRetrievalEvaluator</code>

Information Retrieval

• Evaluated with <code>InformationRetrievalEvaluator</code>

Information Retrieval

• Evaluated with <code>InformationRetrievalEvaluator</code>

<!-- ## Bias, Risks and Limitations *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* --> <!-- ### Recommendations *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* -->

Training Details

Training Dataset

Unnamed Dataset

• Size: 26,299 training samples
• Columns: <code>sentence_0</code> and <code>sentence_1</code>
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1
  type	string	string
  details	<ul><li>min: 16 tokens</li><li>mean: 38.67 tokens</li><li>max: 215 tokens</li></ul>	<ul><li>min: 5 tokens</li><li>mean: 251.42 tokens</li><li>max: 512 tokens</li></ul>

• Samples:

  sentence_0	sentence_1
  <code>What are the key considerations the Commission must evaluate when assessing the feasibility of including municipal waste incineration installations in the EU ETS by 31 July 2026?</code>	<code>By 31 July 2026, the Commission shall present a report to the European Parliament and to the Council in which it shall assess the feasibility of including municipal waste incineration installations in the EU ETS, including with a view to their inclusion from 2028 and with an assessment of the potential need for an option for a Member State to opt out until 31 December 2030. In that regard, the Commission shall take into account the importance of all sectors contributing to emission reductions and potential diversion of waste towards disposal by landfilling in the Union and waste exports to third countries. The Commission shall in addition take into account relevant criteria such as the effects on the internal market, potential distortions</code>
  <code>What are the conditions under which a registrant can withhold certain information from disclosure, and what steps must they take to justify this decision?</code>	<code>NOTES<br><br>Note 1: If it is not technically possible, or if it does not appear scientifically necessary to give information, the reasons shall be clearly stated, in accordance with the relevant provisions.<br><br>Note 2: The registrant may wish to declare that certain information submitted in the registration dossier is commercially sensitive and its disclosure might harm him commercially. If this is the case, he shall list the items and provide a justification.<br><br>▼C1<br><br>INFORMATION REFERRED TO IN ARTICLE 10(a) (i) TO (v)<br><br>1. GENERAL REGISTRANT INFORMATION<br><br>1.1. Registrant<br><br>▼M70<br><br>1.1.1. Name, address, telephone number and email address<br><br>▼C1<br><br>1.1.2. Contact person<br><br>1.1.3. Location of the registrant's production and own use site(s), as appropriate<br><br>▼M70</code>
  <code>What are the specific color indices and chemical identifiers for Pigment Red 112 and Pigment Yellow 14, and what is their respective concentration percentage?</code>	<code>17 (PR17)/CI 12390 229-681-4 6655-84-1 0,1 % Pigment Red 112 (PR112)/CI 12370 229-440-3 6535-46-2 0,1 % Pigment Yellow 14 (PY14)/CI 21095 226-789-3 5468-75-7 0,1 % Pigment Yellow 55 (PY55)/CI 21096 226-789-3 6358-37-8 0,1 % Pigment Red 2 (PR2)/CI 12310 227-930-1 6041-94-7 0,1 % Pigment Red 22 (PR22)/CI 12315 229-245-3 6448-95-9 0,1 % Pigment Red 146 (PR146)/CI 12485 226-103-2 5280-68-2 0,1 % Pigment Red 269 (PR269)/CI 12466 268-028-8 67990-05-0 0,1 % Pigment Orange16 (PO16)/CI 21160 229-388-1 6505-28-8 0,1 % Pigment Yellow 1 (PY1)/CI 11680 219-730-8 2512-29-0 0,1 % Pigment Yellow 12 (PY12)/CI 21090 228-787-8 6358-85-6 0,1 % Pigment Yellow 87 (PY87)/CI 21107:1 239-160-3 15110-84-6, 14110-84-6 0,1 % Pigment Yellow 97 (PY97)/CI 11767</code>

• Loss: <code>MatryoshkaLoss</code> with these parameters:

  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          768,
          512,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 6
• per_device_eval_batch_size: 6
• num_train_epochs: 4
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

<details><summary>Click to expand</summary>

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 6
• per_device_eval_batch_size: 6
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 4
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin

</details>

Training Logs

<details><summary>Click to expand</summary> </details>

Framework Versions

• Python: 3.10.11
• Sentence Transformers: 3.4.1
• Transformers: 4.48.1
• PyTorch: 2.4.0+cu121
• Accelerate: 1.4.0
• Datasets: 3.3.2
• Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

<!-- ## Glossary *Clearly define terms in order to be accessible across audiences.* --> <!-- ## Model Card Authors *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.* --> <!-- ## Model Card Contact *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.* -->