finetune

SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2

This is a sentence-transformers model finetuned from sentence-transformers/all-MiniLM-L6-v2. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/all-MiniLM-L6-v2 <!-- at revision fa97f6e7cb1a59073dff9e6b13e2715cf7475ac9 -->
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity

<!-- - **Training Dataset:** Unknown --> <!-- - **Language:** Unknown --> <!-- - **License:** Unknown -->

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'What was the new name given to the Administrative Charge on June 22, 2022?',
    'Context: customer. Do not tell the customer that the Customer Agreement only allows termination of a contract without an ETF if the change has a material adverse effect on the customer is to the calling plan. If the customer is undecided, the Supervisor/Support Coordinator must apply the credit and note the account accordingly. Refusal to Pay There is no "Refusal to Pay" process for the TUSC. Let customers know that, if they do not pay the surcharge, it continues to show as a balance owed on the account, which can cause interruption of their service for failing to pay billed charges.Table: <table border="1" cellpadding="5" cellspacing="0" width="100%"><tbody><tr><td>\xa0</td><td><b>6/22/22 Changes</b></td><td><b>12/1/22 Changes</b></td></tr><tr><td valign="top"><b>What Changed?</b></td><td valign="top">The Administrative Charge was renamed the Administrative and Telco Recovery Charge.<br/><br/>The charge was increased to $3.30 per voice line.\xa0</td><td valign="top">The charge increased to $1.40 per data-only line.<br/></td></tr><tr><td valign="top"><b>Customer Notifications</b></td><td valign="top">Beginning 5/23/2022 and running through 6/22/2022, the following appeared on all Consumer Bills:<br/><br/>Notice of Changes to Administrative Charge:<br/>Effective June 23, 2022, the monthly Verizon Wireless Administrative and Telco Recovery Charge (formerly the Verizon Wireless Administrative Charge) for voice-capable devices will increase from $1.95 to $3.30 per line. The charge for data-only devices will remain $0.06. The increased charge and updated name will appear on your next bill. For further information regarding the “Admin &amp; Telco Recovery Charge,” review the “Explanation of Surcharges” section of the bill.</td><td>Beginning 11/1/2022 and running through 11/30/2022, the following appeared on all Consumer Bills:<br/><br/>Notice of Changes to Administrative and Telco Recovery Charge:<br/>Effective December 1, 2022, the monthly Verizon Wireless Administrative and Telco Recovery Charge for data-only plans will increase from $0.06 to $1.40 per line. If you have a device on a data-only plan on your account, the increased charge will appear on your next bill. Customers on 5G Home internet and LTE Home Internet plans will not see a monthly increase. The charge for plans with voice service will remain $3.30 per line. For further information regarding the “Admin &amp; Telco Recovery Charge,” review the “Explanation of Surcharges” section of the bill.<br/><br/></td></tr><tr><td valign="top"><b>Positioning for Customer Inquiries</b></td><td valign="top">Mr/Ms Customer,<br/>I want to help you understand the Administrative and Telco Recovery Charge and other fees or surcharges on your bill.<br/>The notification that you received states that the Administrative and Telco Recovery Charge will increase by $1.35 per voice line to $3.30, effective your June bill.\xa0<br/><br/>Verizon reviews the Administrative and Telco Recovery Charge from time to time and makes adjustments to recover some of the administrative and telco expenses and costs of complying with regulatory requirements we incur to provide service.<br/>Other fees included on your bill are listed out by line item on your bill and you can reference them at any time. I would be happy to assist with any additional questions you might have on your bill.</td><td>Mr/Ms Customer,<br/>I want to help you understand the Administrative and Telco Recovery Charge and other fees or surcharges on your bill.<br/>The notification that you received states that the Administrative and Telco Recovery Charge will increase from $0.06 to $1.40 per data-only line, effective your December bill.<br/><br/>Verizon reviews the Administrative and Telco Recovery Charge from time to time and makes adjustments to recover some of the administrative and telco expenses and costs of complying with regulatory requirements we incur to provide service.<br/>Other fees included on your bill are listed out by line item on your bill and you can reference them at any time. I would be happy to assist with any additional questions you<br/>might have on your bill.</td></tr></tbody></table>',
    'To access Payment Hub for Retail Sales: Users do not need to submit an Automated CS Request (ACSR) for basic access. Use the Payment Hub application to submit a Payment Investigation on behalf of the customer. Retail sales can access Payment Hub in the following manner: Search InfoManager for ‘Payment Hub Tool’ My POS > External Services > Payment Hub Note: To report a Payment Hub app error, use either option below: Open a Self-Help Ticket and select category: Payment Hub Call IT Support at 866-899-4872 and a ticket is opened and dispatched to the Payment Hub Support team for resolution.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

<!-- ### Direct Usage (Transformers) <details><summary>Click to see the direct usage in Transformers</summary> </details> --> <!-- ### Downstream Usage (Sentence Transformers) You can finetune this model on your own dataset. <details><summary>Click to expand</summary> </details> --> <!-- ### Out-of-Scope Use *List how the model may foreseeably be misused and address what users ought not to do with the model.* -->

Evaluation

Metrics

Information Retrieval

• Evaluated with <code>InformationRetrievalEvaluator</code>

<!-- ## Bias, Risks and Limitations *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* --> <!-- ### Recommendations *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* -->

Training Details

Training Dataset

Unnamed Dataset

• Size: 4,999 training samples
• Columns: <code>sentence_0</code> and <code>sentence_1</code>
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1
  type	string	string
  details	<ul><li>min: 10 tokens</li><li>mean: 24.57 tokens</li><li>max: 53 tokens</li></ul>	<ul><li>min: 95 tokens</li><li>mean: 229.55 tokens</li><li>max: 256 tokens</li></ul>

• Samples:

  sentence_0	sentence_1
  <code>What is the discount offered for customers who activate or upgrade to a new Smartphone on a Small Business Plan?</code>	<code>Context: who activate or upgrade to a new Smartphone on a Small Business Plan. Upgrade eligible. $5 line access discount for customers who activate or upgrade to a new Smartphone on a Small Business Plan. Upgrade eligible. Upgrade eligible. Offer Options Code Code Description Discount Final Cost Contract Consumer Business Employees N/AAccess Line $5N/A1 Yr N/AAccess Line $5N/A2 Yr Offer Options Code Code Description Discount Final Cost Contract Consumer Business Employees N/AAccess Line $5N/A1 Yr N/AAccess Line $5N/A2 Yr Offer Options Offer Options Code Code Description Discount Final Cost Contract Consumer Business Employees N/AAccess Line $5N/A1 Yr N/AAccess Line $5N/A2 YrTable: <table class="details-table"> <caption></caption> <colgroup> <col class="code"/> <col class="code-description"/> <col class="discount"/> <col class="final-cost"/> <col class="contract"/> <col class="consumer"/> <col class="business"/> <col class="employees"/> </colgroup> <thead> <tr class="row-other"> <th cla...</code>
  <code>What is the line access charge for a smartphone customer under contract who moves from a non-MORE Everything plan to a Small Business Plan after applying the $5 line access discount?</code>	<code>Customers who activate or upgrade to a new smartphone on a Small Business Plan and sign a 1 or 2-year contract automatically receive a $5 line access discount: Available on all Small Business Plan 25, 50, 100 line plans. Available for business customers who are eligible to purchase a Small Business Plan. Smartphone customers under contract moving from a non-MORE Everything plan to a Small Business Plan automatically receive a $5 line access discount. The discount applies to qualifying MDNs while the customer remains on an eligible plan with an eligible device. Note: Smartphone line access charge discounted to $35 (after the $5 line access credit). Customer invoice reflects a $40 line access charge and a $5 promotional credit. The promotion code automatically attaches in all POS systems.</code>
  <code>Who is eligible to receive the $5 discount when changing to a Small Business Plan?</code>	<code>Available for business customers eligible to purchase a Small Business Plan, including: State and Local Government Contracts where legally approved NASPO ValuePoint accounts Note: Discount is automatically applied to new activations, Add a Lines (AALs) and upgrades with a smartphone purchase and a 1 or 2-year contract. Existing customers currently under contract and on a non-MORE Everything plan who complete a plan change to a Small Business Plan automatically receive the $5 discount. Existing smartphone customers under contract and on eligible devices and plans may request the discount. Discount can be applied through all front-end systems and My Business.</code>

• Loss: <code>MultipleNegativesRankingLoss</code> with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 10
• per_device_eval_batch_size: 10
• num_train_epochs: 2
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

<details><summary>Click to expand</summary>

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 10
• per_device_eval_batch_size: 10
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 2
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin

</details>

Training Logs

Framework Versions

• Python: 3.10.12
• Sentence Transformers: 3.3.1
• Transformers: 4.47.1
• PyTorch: 2.5.1+cu121
• Accelerate: 1.2.1
• Datasets: 3.2.0
• Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

<!-- ## Glossary *Clearly define terms in order to be accessible across audiences.* --> <!-- ## Model Card Authors *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.* --> <!-- ## Model Card Contact *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.* -->