custom_bge_baai_cfr

SentenceTransformer based on BAAI/bge-base-en-v1.5

This is a sentence-transformers model finetuned from BAAI/bge-base-en-v1.5. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: BAAI/bge-base-en-v1.5 <!-- at revision a5beb1e3e68b9ab74eb54cfd186867f64f240e1a -->
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Supported Modality: Text

<!-- - **Training Dataset:** Unknown --> <!-- - **Language:** Unknown --> <!-- - **License:** Unknown -->

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'})
  (1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'mean', 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'Placing the health of (or, with respect to a pregnant woman, health of unborn child) in serious there is to a safe to another hospital before delivery; <> may pose a the health or or the unborn child. [ENUM Hospital] includes access as defined section 1861(mm)(1) Act and a emergency hospital as in section 1861(kkk)(2).',
    '<> Placing the health of the individual (or, with respect to a pregnant woman, the health of the woman or her unborn child) in serious jeopardy; <> That there is inadequate time to effect a safe transfer to another hospital before delivery; or <> That transfer may pose a threat to the health or safety of the woman or the unborn child. [ENUM Hospital] includes a critical access hospital as defined in section 1861(mm)(1) of the Act and a rural emergency hospital as defined in section 1861(kkk)(2).',
    '<> If CMS determines that a facility or organization that had previously been determined to be provider-based under this section no longer qualifies for provider-based status, and if the failure to qualify for provider-based status resulted from a material change in the relationship between the provider and the facility or organization that the provider did not report to CMS under paragraph (c) of this section, CMS will take the actions with respect to notice to the provider, adjustment of payments, and continuation of payment described in paragraphs (j)(3), (j)(4), and (j)(5) of this section, and will recover past payments to the provider to the extent described in paragraph (j)(1)(ii) of this section.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.9431, 0.4788],
#         [0.9431, 1.0000, 0.5043],
#         [0.4788, 0.5043, 1.0000]])

<!-- ### Direct Usage (Transformers) <details><summary>Click to see the direct usage in Transformers</summary> </details> --> <!-- ### Downstream Usage (Sentence Transformers) You can finetune this model on your own dataset. <details><summary>Click to expand</summary> </details> --> <!-- ### Out-of-Scope Use *List how the model may foreseeably be misused and address what users ought not to do with the model.* --> <!-- ## Bias, Risks and Limitations *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* --> <!-- ### Recommendations *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* -->

Training Details

Training Dataset

Unnamed Dataset

• Size: 24,712 training samples
• Columns: <code>sentence_0</code> and <code>sentence_1</code>
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1
  type	string	string
  details	<ul><li>min: 23 tokens</li><li>mean: 93.61 tokens</li><li>max: 283 tokens</li></ul>	<ul><li>min: 36 tokens</li><li>mean: 133.76 tokens</li><li>max: 316 tokens</li></ul>

• Samples:

  sentence_0	sentence_1
  <code>Be specially designed to respond medical provide acute to transport sick and and with all State and local laws governing an <> Be equipped emergency warning lights and required by State or laws. Be with telecommunications equipment as required or local law to minimum, one two-way voice radio wireless telephone. <> Be with a oxygen medical equipment as required State or local</code>	<code><> Be specially designed to respond to medical emergencies or provide acute medical care to transport the sick and injured and comply with all State and local laws governing an emergency transportation vehicle. <> Be equipped with emergency warning lights and sirens, as required by State or local laws. <> Be equipped with telecommunications equipment as required by State or local law to include, at a minimum, one two-way voice radio or wireless telephone. <> Be equipped with a stretcher, linens, emergency medical supplies, oxygen equipment, and other lifesaving emergency medical equipment as required by State or local laws.</code>
  <code>Except paragraph (b) this section, a Part D plan sponsor that approves request for expedited determination must notify the enrollee (and the prescribing physician prescriber involved, appropriate) decision, whether adverse or as as the enrollee's condition requires, no [NUM] hours after receiving For the sponsor must notify (and the prescribing physician other prescriber involved, as appropriate) of its determination as expeditiously the enrollee's health condition requires, but later [NUM] hours after of the physician's or other prescriber's supporting statement. If a supporting is not received by end of 14 days from receipt of the exceptions Part D sponsor must notify enrollee prescribing physician involved, appropriate) of expeditiously as the enrollee's condition requires, later [NUM] hours from end of 14 days from receipt of request.</code>	<code>Except as provided in paragraph (b) of this section, a Part D plan sponsor that approves a request for expedited determination must make its determination and notify the enrollee (and the prescribing physician or other prescriber involved, as appropriate) of its decision, whether adverse or favorable, as expeditiously as the enrollee's health condition requires, but no later than 24 hours after receiving the request. For an exceptions request, the Part D plan sponsor must notify the enrollee (and the prescribing physician or other prescriber involved, as appropriate) of its determination as expeditiously as the enrollee's health condition requires, but no later than 24 hours after receipt of the physician's or other prescriber's supporting statement. If a supporting statement is not received by the end of 14 calendar days from receipt of the exceptions request, the Part D plan sponsor must notify the enrollee (and the prescribing physician or other prescriber involved, as appropriate) ...</code>
  <code>subpart implements sections 1902(a)(38), 1903(i)(2), and 1903(n) of Social Security Act. It forth State plan requirements Disclosure by and fiscal and control and of on a provider's other persons of offenses against Medicare, Medicaid, or the title XX services</code>	<code>This subpart implements sections 1124, 1126, 1902(a)(38), 1903(i)(2), and 1903(n) of the Social Security Act. It sets forth State plan requirements regarding— <> Disclosure by providers and fiscal agents of ownership and control information; and <> Disclosure of information on a provider's owners and other persons convicted of criminal offenses against Medicare, Medicaid, or the title XX services program.</code>

• Loss: <code>DenoisingAutoEncoderLoss</code> with these parameters:

  {
      "decoder_name_or_path": "BAAI/bge-base-en-v1.5",
      "need_retokenization": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• num_train_epochs: 10
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

<details><summary>Click to expand</summary>

• overwrite_output_dir: False
• do_predict: False
• prediction_loss_only: True
• per_device_train_batch_size: 8
• per_device_eval_batch_size: 8
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 10
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin
• router_mapping: {}
• learning_rate_mapping: {}

</details>

Training Logs

Training Time

• Training: 1.4 hours

Framework Versions

• Python: 3.12.6
• Sentence Transformers: 5.4.1
• Transformers: 4.56.0
• PyTorch: 2.8.0+cu129
• Accelerate: 1.10.1
• Datasets: 4.8.4
• Tokenizers: 0.22.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

DenoisingAutoEncoderLoss

@inproceedings{wang-2021-TSDAE,
    title = "TSDAE: Using Transformer-based Sequential Denoising Auto-Encoderfor Unsupervised Sentence Embedding Learning",
    author = "Wang, Kexin and Reimers, Nils and Gurevych, Iryna",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2021",
    month = nov,
    year = "2021",
    address = "Punta Cana, Dominican Republic",
    publisher = "Association for Computational Linguistics",
    pages = "671--688",
    url = "https://arxiv.org/abs/2104.06979",
}

<!-- ## Glossary *Clearly define terms in order to be accessible across audiences.* --> <!-- ## Model Card Authors *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.* --> <!-- ## Model Card Contact *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.* -->