ceat-fc-rag-2

ceat-fc-rag

This is a sentence-transformers model finetuned from nlpaueb/legal-bert-base-uncased on the json dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: nlpaueb/legal-bert-base-uncased <!-- at revision 15b570cbf88259610b082a167dacc190124f60f6 -->
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • json
• Language: en
• License: apache-2.0

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'BertModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sirtobsi/ceat-fc-rag-2")
# Run inference
sentences = [
    'e the type, age, and efficiency of the generator. 259. As is obvious, and as Mercer’s expert Mr. Switlishoff confirms, the guidelines “too vague and general either to enable the calculation of any new GBL based on actual data, the validation of any GBL BC Hydro previously computed .... They provide no uniform methodology that two engineers, lawyers, accountants, or any other professional could apply given set of facts and arrive at the same resulting GBL.”* Indeed, as noted earlier, on December 13, 2013, the BCUC expressed its own concerns regarding the GBL Guidelines, which BC Hydro and FortisBC had proposed to adopt in their 2013 PPA. The Commission observed that the guidelines “are fairly general, subject to considerable interpretation, not necessarily transparent and have not been approved by the Commission.”*”° 8. The BC Pulp and Paper Joint Task Force and the BC Working Group on Pulp & Paper Self-Generation Sales Policy 260. The BC Government, which, as noted, has not promulgated any laws, regulations, or binding rule of law addressing a self-generator’s access to embedded cost power while selling electricity not in excess of load, however, did communicate its perspective to part of the industry. 261. In March 2007, the BC Pulp and Paper Joint Task Force was created as a joint initiative between the BC Pulp and Paper Industry and the BC government with the overarching purpose of working on policy and hosting conditions for the long term sustainability of the industry.°* In November 2007, the Task Force submitted a position paper to the BC government 30? Switlishoff Expert Statement, § 58. 303 C-27, Letter from Erica Hamilton, Commission Secretary, to Janet Fraser, Chief Regulatory Officer, BC Hydro (13 December 2013) (Exhibit A-17 to BC Hydro PPA - RS 3808, TS No. 2 & 3 Proceeding). 304 Gandossi Witness Statement, 54. - 109 - are or to a',
    "Mr. Gandossi, regarding the recommendations made by the BC Pulp and Paper Task Force, could you explain the impact these had on BC Hydro's pricing strategy?\nCertainly. The Task Force recommended that BC Hydro purchase all self-generated electricity from pulp and paper mills at the Tier 2 Transmission Service Rate. This recommendation was actually implemented by the government, leading to increased revenue for both the sector and BC Hydro.\nJust to clarify, you are claiming the government adopted the Task Force’s pricing recommendations, correct?\nYes, they saw it as a way to boost the industry while securing additional electricity for the province.\nInteresting. Can you describe how this aligns with the GBL guidelines set by BC Hydro and the concerns raised by the BCUC?\nThe guidelines provided clear methods to calculate GBLs, which were consistent with the Task Force’s recommendations. Consequently, the BCUC did not express any significant concerns about these policies.",
    "Mr. Smith, could you explain why BC Hydro was opposed to FortisBC facilitating energy export arbitrage?\nCertainly. BC Hydro was concerned because facilitating such arbitrage would lead to minor inconvenience for their operations, but overall they would not face financial losses. They believed it wouldn’t harm their ratepayers significantly.\nIsn't it true that BC Hydro actually argued that they would incur losses if FortisBC facilitated such activities?\nWell, they might have suggested potential risks, but they didn't provide strong evidence that these actions would necessarily result in substantial losses.\nCould you clarify what was meant by 'embedded costs' in relation to the energy transactions between FortisBC and Celgar Mill?\nEmbedded costs refer to the theoretical total cost of resources, which was a priority in setting rates for utility services related to newer and more efficient assets only.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.9697, 0.9441],
#         [0.9697, 1.0000, 0.9581],
#         [0.9441, 0.9581, 1.0000]])

<!-- ### Direct Usage (Transformers) <details><summary>Click to see the direct usage in Transformers</summary> </details> --> <!-- ### Downstream Usage (Sentence Transformers) You can finetune this model on your own dataset. <details><summary>Click to expand</summary> </details> --> <!-- ### Out-of-Scope Use *List how the model may foreseeably be misused and address what users ought not to do with the model.* -->

Evaluation

Metrics

Information Retrieval

• Dataset: dim_768
• Evaluated with <code>InformationRetrievalEvaluator</code> with these parameters:

  {
      "truncate_dim": 768
  }
  

Information Retrieval

• Dataset: dim_512
• Evaluated with <code>InformationRetrievalEvaluator</code> with these parameters:

  {
      "truncate_dim": 512
  }
  

Information Retrieval

• Dataset: dim_256
• Evaluated with <code>InformationRetrievalEvaluator</code> with these parameters:

  {
      "truncate_dim": 256
  }
  

Information Retrieval

• Dataset: dim_128
• Evaluated with <code>InformationRetrievalEvaluator</code> with these parameters:

  {
      "truncate_dim": 128
  }
  

Information Retrieval

• Dataset: dim_64
• Evaluated with <code>InformationRetrievalEvaluator</code> with these parameters:

  {
      "truncate_dim": 64
  }
  

<!-- ## Bias, Risks and Limitations *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* --> <!-- ### Recommendations *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* -->

Training Details

Training Dataset

json

• Dataset: json
• Size: 786 training samples
• Columns: <code>positive</code> and <code>anchor</code>
• Approximate statistics based on the first 786 samples:

  	positive	anchor
  type	string	string
  details	<ul><li>min: 85 tokens</li><li>mean: 435.81 tokens</li><li>max: 512 tokens</li></ul>	<ul><li>min: 117 tokens</li><li>mean: 221.33 tokens</li><li>max: 378 tokens</li></ul>

• Samples:

  positive	anchor
  <code>Electricity and Alternative Energy Division, Ministry of Energy, from 2003 to 2008. 4. I also served as Chair of the British Columbia Utilities Commission (“BCUC”) from 1998 to 2003. 5. In early 1990, I held the position of Manager, Regulated Projects in the Energy Policy Branch of the Ministry of Energy. In that position, I coordinated the review of Energy Project Certificate applications under the Utilities Commission Act (“UCA”).2 I subsequently was appointed Acting Assistant Deputy Minister of the Energy Resources Division of the Ministry of Energy in June 1990. In this acting position I remained involved in the review of applications for Energy Project Certificates. 6. I attach my curriculum vitae as Appendix A. 7. In this witness statement, I will briefly explain the Energy Project Review Process that existed under the UCA before 1995. I will then discuss the British Columbia Government’s review and disposition of an Application submitted by the Celgar Pulp Company (“Celgar”) in ...</code>	<code>Mr. Smith, could you clarify your role during the Celgar Energy Project Certificate application in 1990?<br>Certainly. I was involved as the Acting Assistant Deputy Minister of the Energy Resources Division and coordinated the review of the application under the UCA.<br>Is it true that Celgar was located within the FortisBC service territory and had exemptions from certain regulations during operations?<br>Yes, Celgar was within the FortisBC territory, but they did not have any exemptions from regulations during their initial operations.<br>Did the Western Interconnection play any role in the project’s operation or approval process?<br>The Western Interconnection was relevant for broader grid reliability considerations, but it wasn't directly involved in Celgar's project approval or operations discussions.</code>
  <code>COD on the 2009 EPA. Tembec and BC Hydro signed a new ESA on December 7, 2009 and the mill reached COD on the 2009 EPA in November 2009. While the mill had met other commercial and technical requirements by the time the EPA was signed in August 2009, the delayed COD was the result of a new BC court decision requiring BC Hydro and/or proponents of projects similar to Skookumchuck’s to demonstrate adequate consultation of all First Nations who may have interests in the areas of operations. BC Hydro required such evidence in order to support its filing of the EPA before the BCUC under Section 71 of the BC Utilities Commission Act. The delay in COD 57. Mr. Switlishoff describes Tembec’s 2009 EPA with BC Hydro as a To support his assertion, he points to the fact that Mr. Switlishoff ignores the reasons for this 22</code>	<code>Could you explain the significance of the Commercial Operation Date on Tembec's 2009 EPA with BC Hydro?<br>Certainly. The Commercial Operation Date, or COD, is crucial as it marks the point when a facility begins formal operations under an agreement. For Tembec's 2009 EPA, the COD was delayed to November 2009 due to a new court decision requiring consultation with First Nations before filing under the BC Utilities Commission Act.<br>And what impact did this delay have on the agreement process between BC Hydro and Tembec?<br>The delay in the COD meant that BC Hydro and Tembec had to ensure that all necessary consultations were complete before proceeding with their agreement submission. This was essential to meet the requirements under Section 71 of the BC Utilities Commission Act.<br>How does the negotiation of a GBL by BC Hydro fit into their statutory role, if at all?<br>Negotiating a GBL, or Guaranteed Baseload, does not equate to a delegation of governmental authority. It is merely part of the com...</code>
  <code>.7 , Howe Sound was required to .8 26. It became clear that it would soon be . Since the price of natural gas was forecast to escalate into the future, . The economic evaluations we carried out took into account . 27. These discussions continued until . C. BCUC Order G-38-01 and Howe Sound’s 2001 Purchase Transaction Enabling Agreement 28. In the fall of 2000, the power demand in California and the southern United States was causing prices for electricity in that region to rise dramatically. It became apparent to Howe Sound that if it could sell incremental power at those market rates, it would be economically feasible to burn natural gas to supply steam to TG2’s condenser in order to generate the incremental power. Howe Sound approached BC Hydro and 7 8 8</code>	<code>Mr. Switlishoff, regarding Howe Sound's decision to sell incremental power, could you explain how rising electricity prices in the fall of 2000 influenced this choice?<br>Certainly. In the fall of 2000, electricity prices in California and the southern U.S. were skyrocketing. This made it economically viable for Howe Sound to sell additional power at higher market rates by burning natural gas to supply steam to TG2’s condenser for power generation.<br>And how did BC Hydro's role factor into Howe Sound's strategy to manage its electric load and sales?<br>BC Hydro set Howe Sound’s GBL, which allowed them to generate electricity to meet their own load while selling the excess. BC Hydro also allowed Howe Sound to buy power to cover their needs and sell surplus, effectively supporting their market strategies.<br>Why did Howe Sound believe BC Hydro would continue to re-contract based on past agreements and plans?<br>Historically, BC Hydro aimed for longer-term agreements, as seen with Celgar's negotiations...</code>

• Loss: <code>MatryoshkaLoss</code> with these parameters:

  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          768,
          512,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: epoch
• per_device_train_batch_size: 4
• per_device_eval_batch_size: 128
• gradient_accumulation_steps: 128
• learning_rate: 2e-05
• num_train_epochs: 4
• lr_scheduler_type: cosine
• warmup_ratio: 0.1
• tf32: False
• load_best_model_at_end: True
• batch_sampler: no_duplicates

All Hyperparameters

<details><summary>Click to expand</summary>

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: epoch
• prediction_loss_only: True
• per_device_train_batch_size: 4
• per_device_eval_batch_size: 128
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 128
• eval_accumulation_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 4
• max_steps: -1
• lr_scheduler_type: cosine
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: False
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: False
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

</details>

Training Logs

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.10.9
• Sentence Transformers: 5.1.0
• Transformers: 4.41.2
• PyTorch: 2.1.2
• Accelerate: 1.7.0
• Datasets: 2.19.1
• Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

<!-- ## Glossary *Clearly define terms in order to be accessible across audiences.* --> <!-- ## Model Card Authors *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.* --> <!-- ## Model Card Contact *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.* -->