blending-embedder

SentenceTransformer based on BAAI/bge-base-en-v1.5

This is a sentence-transformers model finetuned from BAAI/bge-base-en-v1.5. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: BAAI/bge-base-en-v1.5 <!-- at revision a5beb1e3e68b9ab74eb54cfd186867f64f240e1a -->
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Supported Modality: Text

<!-- - **Training Dataset:** Unknown --> <!-- - **Language:** Unknown --> <!-- - **License:** Unknown -->

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'})
  (1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'cls', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'Represent this sentence for searching relevant passages: How can a blending engineer distinguish between filter-clogging sediments caused by the solvent cleaning action of high-concentration biodiesel versus those resulting from fuel degradation over time?',
    'not mimic any condition the fuel might encounter in use, but is designed to force rapid oxidation, which differentiates stable and unstable fuels. Results from this test provide a relative stability index; longer times mean more stable fuel. In some cases, deposits from the cleaning or solvent effect of B100 have been confused with gums and sediments that could form in storage as the B100 ages. Although sediment can clog a filter in either case, care should be taken to make sure the reason for the clogging is properly identified. For example, if oxidation stability and acid number are within specification, sediments are most likely due to the cleaning effect and not to aging or oxidation. Guidelines to help identify',
    'with petroleum home heating oil. • Antioxidants, whether natural or incorporated as additives, can significantly increase the storage life or stability of B100. The use of antioxidant additives is highly recommended with B100, especially with longer-term storage. Actual results and improvements may vary depending upon the chemistry of each additive. • Limiting or preventing biodiesel contact with air reduces or eliminates fuel oxidation and increases storage life. Commercially, this is done by using a nitrogen blanket on storage tanks or storing biodiesel in sealed drums or totes with minimal headspace. Accelerated stability tests like EN15751 will show if a fuel has high or low stability, but this cannot exactly predict the amount of time a fuel can be stored. The',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.5562, 0.0880],
#         [0.5562, 1.0000, 0.4385],
#         [0.0880, 0.4385, 1.0000]])

<!-- ### Direct Usage (Transformers) <details><summary>Click to see the direct usage in Transformers</summary> </details> --> <!-- ### Downstream Usage (Sentence Transformers) You can finetune this model on your own dataset. <details><summary>Click to expand</summary> </details> --> <!-- ### Out-of-Scope Use *List how the model may foreseeably be misused and address what users ought not to do with the model.* --> <!-- ## Bias, Risks and Limitations *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* --> <!-- ### Recommendations *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* -->

Training Details

Training Dataset

Unnamed Dataset

• Size: 371 training samples
• Columns: <code>sentence_0</code> and <code>sentence_1</code>
• Approximate statistics based on the first 100 samples:

  	sentence_0	sentence_1
  type	string	string
  modality	text	text
  details	<ul><li>min: 23 tokens</li><li>mean: 35.15 tokens</li><li>max: 56 tokens</li></ul>	<ul><li>min: 133 tokens</li><li>mean: 167.69 tokens</li><li>max: 238 tokens</li></ul>

• Samples:

  sentence_0	sentence_1
  <code>Represent this sentence for searching relevant passages: What feedstocks are commonly used to produce biodiesel?</code>	<code>is also provided. Biodiesel is produced from plant oils, animal fats, recycled cooking oils, and/or greases and has several advantages: • Renewable, with a much lower carbon intensity/life cycle greenhouse gas emissions profile than petroleum diesel fuel, such that it reduces global warming emissions. • Can be used at up to at least 20% blend level in most diesel equipment with zero or minor modifications. • Compatible with all new on- and off-road diesel engines—often called “new technology diesel engines” (NTDEs)—and burners and backward-compatible with most equipment. • Demonstrated to reduce tailpipe soot emissions and other criteria pollutants from older (e.g., legacy) engines without modern emission controls, as well as from industrial boiler and home heating systems. • Nontoxic, biodegradable,</code>
  <code>Represent this sentence for searching relevant passages: What kinds of residual substances in an unwashed transport vessel would be considered unacceptable when loading biodiesel?</code>	<code>B100 must be transported in a way that does not lead to contamination. The following procedures are recommended for trucks and railcars and are used by distributors and transporters of diesel fuel and/or biodiesel: • Ensure trucks and railcars are constructed of aluminum, carbon steel, or stainless steel. • Ensure proper inspection or washout (i.e., a washout certificate) before loading. • Check for previous load carried and residual material. Generally, only diesel fuel or biodiesel is acceptable as a residual. If the vessel has not gone through a washout, some residuals—including food products or raw plant oils, gasoline, or lubricants—may not be acceptable. • Ensure there is no residual water in the tank. • Check that hoses and seals are</code>
  <code>Represent this sentence for searching relevant passages: What characteristic of the fatty acid chains in biodiesel feedstocks explains why energy content remains relatively consistent regardless of the source oil or fat used?</code>	<code>No. 2 diesel fuel. Due to the similar carbon chain length number of the fatty acid chains in the oils and fats used to produce biodiesel, the energy content of biodiesel is similar, regardless of the feedstock and the particular biodiesel process. The energy content of biodiesel and diesel fuel blend linearly, so the energy content of B50 would be midway between the energy content of the starting biodiesel and diesel fuel. However, these differences in energy content do not inherently translate into lower biodiesel fuel economy, especially with engines that employ modern diesel particulate filter/selective catalytic reduction technology that can be positively impacted by lower particulate emissions of biodiesel. Typical reference values for various fuels and biodiesel are shown</code>

• Loss: <code>MultipleNegativesRankingLoss</code> with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false,
      "directions": [
          "query_to_doc"
      ],
      "partition_mode": "joint",
      "hardness_mode": null,
      "hardness_strength": 0.0
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• multi_dataset_batch_sampler: round_robin

All Hyperparameters

<details><summary>Click to expand</summary>

• per_device_train_batch_size: 8
• num_train_epochs: 3
• max_steps: -1
• learning_rate: 5e-05
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_steps: 0
• optim: adamw_torch_fused
• optim_args: None
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• optim_target_modules: None
• gradient_accumulation_steps: 1
• average_tokens_across_devices: True
• max_grad_norm: 1
• label_smoothing_factor: 0.0
• bf16: False
• fp16: False
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• use_liger_kernel: False
• liger_kernel_config: None
• use_cache: False
• neftune_noise_alpha: None
• torch_empty_cache_steps: None
• auto_find_batch_size: False
• log_on_each_node: True
• logging_nan_inf_filter: True
• include_num_input_tokens_seen: no
• log_level: passive
• log_level_replica: warning
• disable_tqdm: False
• project: huggingface
• trackio_space_id: None
• trackio_bucket_id: None
• trackio_static_space_id: None
• per_device_eval_batch_size: 8
• prediction_loss_only: True
• eval_on_start: False
• eval_do_concat_batches: True
• eval_use_gather_object: False
• eval_accumulation_steps: None
• include_for_metrics: []
• batch_eval_metrics: False
• save_only_model: False
• save_on_each_node: False
• enable_jit_checkpoint: False
• push_to_hub: False
• hub_private_repo: None
• hub_model_id: None
• hub_strategy: every_save
• hub_always_push: False
• hub_revision: None
• load_best_model_at_end: False
• ignore_data_skip: False
• restore_callback_states_from_checkpoint: False
• full_determinism: False
• seed: 42
• data_seed: None
• use_cpu: False
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• dataloader_prefetch_factor: None
• remove_unused_columns: True
• label_names: None
• train_sampling_strategy: random
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• ddp_static_graph: None
• ddp_backend: None
• ddp_timeout: 1800
• fsdp: None
• fsdp_config: None
• deepspeed: None
• debug: []
• skip_memory_metrics: True
• do_predict: False
• resume_from_checkpoint: None
• warmup_ratio: None
• local_rank: -1
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin
• router_mapping: {}
• learning_rate_mapping: {}

</details>

Training Time

• Training: 30.2 minutes

Framework Versions

• Python: 3.12.3
• Sentence Transformers: 5.6.0
• Transformers: 5.12.1
• PyTorch: 2.12.1+cpu
• Accelerate: 1.14.0
• Datasets: 5.0.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{oord2019representationlearningcontrastivepredictive,
      title={Representation Learning with Contrastive Predictive Coding},
      author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
      year={2019},
      eprint={1807.03748},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/1807.03748},
}

<!-- ## Glossary *Clearly define terms in order to be accessible across audiences.* --> <!-- ## Model Card Authors *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.* --> <!-- ## Model Card Contact *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.* -->