bge-small-finetuned

BGE Small English v1.5

This is a sentence-transformers model finetuned from BAAI/bge-small-en-v1.5. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: BAAI/bge-small-en-v1.5 <!-- at revision 5c38ec7c405ec4b44b94cc5a9bb96e735b38267a -->
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity

<!-- - **Training Dataset:** Unknown -->

• Language: en
• License: apache-2.0

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True, 'architecture': 'BertModel'})
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("MistyDragon/bge-small-finetuned")
# Run inference
sentences = [
    'search_document: 386\u2003 ◾\u2003 Production and Operations Management Systems\n10.8  Location Decisions Using the Transportation \nModel\nTransportation costs are a primary concern for a new start-up company or division. \nThis also applies to an existing company that intends to relocate. Finally, it should \nbe common practice to reevaluate the current location of an ongoing business so \nthat the impact of changing conditions and new opportunities are not overlooked. \nWhen shipping costs are critical for the location decision, the transportation model \n(TM) can determine minimum cost or maximum profit solutions that specify opti-\nmal shipping patterns between many locations.\nTransportation costs include the combined costs of moving raw materials to \nthe plant and of transporting finished goods from the plant to one or more ware -\nhouses. It is easier to explain the TM with the following numerical example than \nwith abstract math equations. A doll manufacturer has decided to build a fac -\ntory in the center of the United States. More specifically, Missouri and Ohio are \nidentified as the potential states. Several sites in the two regions have been identi -\nfied. Two cities have been chosen as candidates. These are St Louis, Missouri, and \nColumbus, Ohio. Real-estate costs are about equal in both. The problem is to \nselect one of the two cities. The decision will be based on the shipping (transporta -\ntion) costs.\n10.8.1 Shipping (Transportation or Distribution) Costs\nThe average cost of shipping (also known as the cost of distribution or cost of trans-\nportation) the components that the company uses to the Columbus, Ohio, location \nis $6 per production unit. Shipping costs average only $3 per unit to St Louis, \nMissouri. In TM terminology, shippers (suppliers, in this case) are called sources or \norigins. Those receiving shipments (producers, in this case) are called destinations.\nThe average cost of shipping from the Columbus, Ohio, location to the \n market—distributor’s warehouse is $2 per unit. The average cost of shipping from \nSt Louis, Missouri, to the market—distributor’s warehouse is $4 per unit. The same \nterminology applies. The shipper is the producer (source or origin) and the receivers \nare the distributors or customers (destinations). The configuration of origins and \ndestinations are shown in Figure 10.1.\nTotal transportation costs to and from the Columbus, Ohio, plant are \n$6 + $2 = $8 per unit; for St Louis, Missouri, they are $3 + $4 = $7. Other things \nbeing equal, the company should choose St Louis, Missouri. However, the real \nworld is not as simple as this.\nThe problem becomes more complex when there are a number of origins com -\npeting for shipments to a number of destinations. We will illustrate the com -\nplexity of the problem and its solution using the example of Rukna Auto Parts \nManufacturing Company.',
    'search_query: In the context of the Transportation Model (TM), what are the primary considerations for a company when deciding on a new location for its operations?',
    'search_query: What is the primary objective of loading in the production scheduling process?',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.7613, 0.4329],
#         [0.7613, 1.0000, 0.4239],
#         [0.4329, 0.4239, 1.0000]])

<!-- ### Direct Usage (Transformers) <details><summary>Click to see the direct usage in Transformers</summary> </details> --> <!-- ### Downstream Usage (Sentence Transformers) You can finetune this model on your own dataset. <details><summary>Click to expand</summary> </details> --> <!-- ### Out-of-Scope Use *List how the model may foreseeably be misused and address what users ought not to do with the model.* -->

Evaluation

Metrics

Information Retrieval

• Dataset: dim_384
• Evaluated with <code>InformationRetrievalEvaluator</code> with these parameters:

  {
      "truncate_dim": 384
  }
  

<!-- ## Bias, Risks and Limitations *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* --> <!-- ### Recommendations *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* -->

Training Details

Training Dataset

Unnamed Dataset

• Size: 525 training samples
• Columns: <code>positive</code> and <code>anchor</code>
• Approximate statistics based on the first 525 samples:

  	positive	anchor
  type	string	string
  details	<ul><li>min: 11 tokens</li><li>mean: 432.37 tokens</li><li>max: 512 tokens</li></ul>	<ul><li>min: 7 tokens</li><li>mean: 30.53 tokens</li><li>max: 103 tokens</li></ul>

• Samples:

  positive	anchor
  <code>search_document: 9192 0.9207 0.9222 0.9236 0.9215 0.9265 0.9279 0.9292 0.9306 0.9319<br>1.5 0.9332 0.9345 0.9357 0.9370 0.9382 0.9394 0.9406 0.9418 0.9492 0.9441<br>1.6 0.9452 0.9463 0.9474 0.9484 0.9495 0.9505 0.9515 0.9525 0.9535 0.9545<br>1.7 0.9554 0.9564 0.9573 0.9582 0.9591 0.9599 0.9608 0.9616 0.9625 0.9633</code>	<code>search_query: What is the value of the function at x = 1.5?</code>
  <code>search_document: 72  •  Quality   Management:   Theory   and   Applicatio n<br>secondary school, or gymnasium. Tertiary education normally includes <br>undergraduate and postgraduate education, as well as vocational educa -<br>tion and training. Colleges and universities are the main institutions that <br>provide tertiary education. Tertiary education generally results in the <br>receipt of certificates, diplomas, or academic degrees.<br>Higher education includes the teaching, research, and social services <br>activities of universities, and within the realm of teaching, it includes <br>both<br> <br>the undergraduate level (sometimes referred to as tertiary education) <br>and the graduate (or postgraduate) level (sometimes referred to as gradu-<br>ate school). Higher education in the United States and Canada generally <br>involves work toward a degree-level or foundation degree qualification. <br>In most developed countries, a high proportion of the population (up to <br>50<br> <br>percent) now enters higher education at some time in t...</code>	<code>search_query: What is the primary difference between tertiary and higher education as described in the document?</code>
  <code>search_document: 273<br>Chapter 8<br>Quality Management<br>Readers’ Choice—“Quality means doing it <br>right when no one is looking.”—Henry Ford<br>Apte, U.M., and Reynolds, C.C., Quality Management at <br>Kentucky Fried Chicken, Interfaces, 25(3), 1995, p. 6. The pro-<br>gram developed by Kentucky Fried Chicken (KFC) Corp. to <br>improve service quality is used as a benchmark for continuous <br>process improvement by all KFC stores. The reduced service <br>time as a result of this program is one of the measurements of <br>quality.<br>Crosby, P.B., Quality is Free (The Art of Making Quality <br>Certain). McGraw-Hill, 1979. Crosby (1979) demanded a zero-<br>defects goal which treats any failures as intolerable.<br>Harris, C.R., and Yit, W., Successfully Implementing Statistical <br>Process Control in Integrated Steel Companies, Interfaces, 24(5), <br>1994, p. 49. Implementation processes of statistical process con-<br>trol (SPC) projects were analyzed at 12 integrated steel compa-<br>nies to identify key success (and failure) factors.<br>Hossein...</code>	<code>search_query: In the context of the document, which company developed a program to improve service quality that is used as a benchmark for continuous process improvement by all KFC stores?</code>

• Loss: <code>MultipleNegativesRankingLoss</code> with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: epoch
• per_device_eval_batch_size: 16
• gradient_accumulation_steps: 8
• learning_rate: 2e-05
• num_train_epochs: 4
• lr_scheduler_type: cosine
• warmup_ratio: 0.1
• bf16: True
• tf32: False
• load_best_model_at_end: True
• optim: adamw_torch_fused
• push_to_hub: True
• hub_model_id: MistyDragon/bge-small-finetuned
• push_to_hub_model_id: bge-small-finetuned
• batch_sampler: no_duplicates

All Hyperparameters

<details><summary>Click to expand</summary>

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: epoch
• prediction_loss_only: True
• per_device_train_batch_size: 8
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 8
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 4
• max_steps: -1
• lr_scheduler_type: cosine
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: False
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: True
• resume_from_checkpoint: None
• hub_model_id: MistyDragon/bge-small-finetuned
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: bge-small-finetuned
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

</details>

Training Logs

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.10.12
• Sentence Transformers: 5.0.0
• Transformers: 4.53.1
• PyTorch: 2.7.1+cu126
• Accelerate: 1.8.1
• Datasets: 3.6.0
• Tokenizers: 0.21.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

<!-- ## Glossary *Clearly define terms in order to be accessible across audiences.* --> <!-- ## Model Card Authors *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.* --> <!-- ## Model Card Contact *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.* -->