EnergyEmbed-v1

SentenceTransformer based on Alibaba-NLP/gte-multilingual-base

This is a sentence-transformers model finetuned from Alibaba-NLP/gte-multilingual-base on the offshore_energy_v1 dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: Alibaba-NLP/gte-multilingual-base <!-- at revision 9bbca17d9273fd0d03d5725c7a4b0f6b45142062 -->
• Maximum Sequence Length: 8192 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • offshore_energy_v1

<!-- - **Language:** Unknown --> <!-- - **License:** Unknown -->

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False, 'architecture': 'NewModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("Sampath1987/EnergyEmbed-v1")
# Run inference
sentences = [
    'What occupational health hazards are anticipated with large construction projects during the energy transition?',
    'endotoxins and fungi. The authors recommended that\nongoing real–time measurement of these exposures be\ncarried out to identify boundary conditions, phases, and\nsettings with the highest pollutant release.  \n12 — Health in the energy transition  \nGood quality studies are needed on the health effects of\nrenewable energy sources. Such studies should include\npopulations and patients with well-characterized exposure,\nhigh-quality information on outcome, and assessment of\npotential confounders. While retrospective (e.g., case-control)\nstudies might produce useful results, prospective longitudinal\nstudies would provide the strongest evidence.  \nSeveral LCA studies have been conducted for the different\ntechnologies. These LCAs reported relative low levels of\nemissions during the lifecycle of renewable sources of\nenergy. Few of these studies included a comparison with\nfossil-based technologies. When more life cycle studies\nbecome available it would be important to include them\nin the literature review. While looking at the life cycle of a\ncertain technology, other health effects in the value chain\ncould potentially be identified (reference: UNECE on Carbon\nNeutrality in the UNECE Region: Integrated Life-cycle\nAssessment of Electricity Sources).  \nAs of December 2024, very few occupational and public\nhealth hazards specific to energy transition technologies\nhave been identified. The energy transition is in an early stage\nand will evolve quickly, and additional hazards unique to\nenergy transition activities may emerge; the specifics of this\nare, at this time, uncertain.  \nWhat is certain is that the energy transition will involve large\nconstruction projects whose risks (and effective methods to\nmanage those risks) are well-known and understood. Existing\noccupational health approaches will be able to manage\nthese risks effectively, provided the correct assessments are\nconducted properly.',
    'institutionalized political structures to realize particular social objectives or serve particular\nconstituencies.  \n**Non-hazardous waste:** Waste, other than Hazardous waste, resulting from company\noperations, including process and oil field wastes disposed of, on site or off site, as well as\noffice, commercial or packaging related wastes [ENV-7].  \n**Normalization:** The ratio of a quantitative indicator output (e.g. emissions) to an\naggregated measure of another output (e.g. oil and gas production or refinery throughput)  \n[Module 1 _Reporting process_ ].  \n**Occupational illness:** An Employee or Contractor health condition or disorder requiring\nmedical treatment due to a workplace Incident, typically involving multiple exposures to\nhazardous substances or to physical agents. Examples include noise-induced hearing loss,\nrespiratory disease, and contact dermatitis [SHS-3].  \n**Occupational injury:** Harm of an Employee or Contractor resulting from a single\ninstantaneous workplace incident that results in medical treatment (beyond simple first aid),\nwork restrictions, days away from work (lost time) or a Fatality [SHS-3].  \n**Operating area:** An area where business activities take place with potential to interact with\nthe adjacent environment [ENV-4].  \n**Operation:** A generic term used to denote any kind of business activity involving productrelated processes, such as production, manufacturing and transport. Note: the term oil and\ngas operations used in the Guidance is intended to be broad and inclusive of other types of\nproduct, such as chemicals.  \n**7.5**',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.5613, 0.2285],
#         [0.5613, 1.0000, 0.2283],
#         [0.2285, 0.2283, 1.0000]])

<!-- ### Direct Usage (Transformers) <details><summary>Click to see the direct usage in Transformers</summary> </details> --> <!-- ### Downstream Usage (Sentence Transformers) You can finetune this model on your own dataset. <details><summary>Click to expand</summary> </details> --> <!-- ### Out-of-Scope Use *List how the model may foreseeably be misused and address what users ought not to do with the model.* -->

Evaluation

Metrics

Triplet

• Dataset: ai-job-validation
• Evaluated with <code>TripletEvaluator</code>

<!-- ## Bias, Risks and Limitations *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* --> <!-- ### Recommendations *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* -->

Training Details

Training Dataset

offshore_energy_v1

• Dataset: offshore_energy_v1 at 4e9339c
• Size: 53,913 training samples
• Columns: <code>anchor</code>, <code>positive</code>, and <code>negative</code>
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	<ul><li>min: 14 tokens</li><li>mean: 23.77 tokens</li><li>max: 42 tokens</li></ul>	<ul><li>min: 36 tokens</li><li>mean: 392.08 tokens</li><li>max: 961 tokens</li></ul>	<ul><li>min: 45 tokens</li><li>mean: 389.63 tokens</li><li>max: 1109 tokens</li></ul>

• Samples:

  anchor	positive	negative
  <code>What statistical methods were employed to enhance the accuracy of comparisons in the field testing of shaped cutters?</code>	<code>As shaped polycrystalline diamond compact (PDC) cutter geometries become more prevalent across the industry, this paper statistically reviews field testing of novel shaped PDC cutters in a variety of challenging applications. Firstly, the paper identifies the improvement in efficiency when compared with conventional PDC cutter geometries. Secondly, it confirms the reliability and robustness of the aforementioned shaped cutter geometries.<br>After several years of field testing shaped PDC cutter geometries, the question of how they hold up against conventional cylinder-shaped cutters remains unanswered. This study looks at drill bits that have the same overall design; however, each bit has different shape configurations that are deployed in a range of hole sizes and drilling applications. Data was collected from more than 100 runs and included advanced dull evaluation techniques, data mining, and comparative analyses. During data collation and interpretation, several statistical methods we...</code>	<code>This paper details the improvements to drilling performance and torsional response of fixed cutter bits when changing from a conventional 19-mm cutter diameter configuration to 25-mm cutter diameters for similar blade counts in two different hole sizes. Key performance metrics include rate of penetration (ROP), rerun-ability, torsional response, and ability to maintain tool-face control during directional drilling.<br>A high-performance drilling application was selected with several existing offset wells using a 12¼-in., five-bladed, 19-mm (519) drill bit design, and a concept bit developed using 25-mm diameter cutters while maintaining comparable ancillary features. This was tested in the same field on both vertical and S-shape sections using the same bent-housing motor assembly and drilling performance compared to the existing offsets. A 17½-in. hole size application that experiences high drillstring vibration was also selected, and a 25-mm cutter diameter drill bit was designed with co...</code>
  <code>What are vapor recovery units (VRU) used for in oil and gas operations?</code>	<code>## 4. Vapour recovery units <br>Vapor recovery units (VRU) are used to prevent emissions by capturing the streams and<br>re-routing them either back to the process or for use as fuel. More details on the<br>components, installation, and operation of VRU are captured in the following sections.</code>	<code>##### 3.1.2 Reduction and recovery of glycol dehydration flash gas <br>Gas from the flash vessel will consist primarily of hydrocarbons and is continuously<br>produced. If installed, a flash vessel will typically remove 90% or more of the entrained<br>hydrocarbon gas and dissolved gases in the glycol leaving the contactor column. <br>Glycol flash vessels typically operate at 3-7 barg [18], meaning there is generally a sufficient<br>pressure drop for the flash gas to commonly be routed to flare or a low-pressure fuel gas<br>system. If the composition of the flash gas prevents this, or there is no fuel gas system,<br>then a Vapour Recovery Unit (VRU) may be needed for recovery into other process units. <br>Minimization of the flash gas itself is also possible by optimizing the glycol flowrate,<br>such as by adjusting the dry gas water temperature specification based on accurate site<br>conditions because the water dew point needed could vary seasonally or from site to<br>site by using more accurate ambient temperatur...</code>
  <code>What challenges are posed by fractures and faults in the completion of MRC wells?</code>	<code>The Maximum Reservoir Contact (MRC) concept was developed to improve well productivity and sustainability by maximizing the contact area with target reservoirs. MRC is a proven technology for the development of tight/non-economical reservoirs. Completion design for MRC wells plays a vital role in enhancing well deliverability, monitoring and accessibility.<br>MRC technology was put into application to appraise a tight and thin heterogeneous carbonate reservoir in a giant offshore field in Abu Dhabi. Different completion scenarios were simulated to select the best suited completion to achieve enhanced well deliverability, monitoring and accessibility.<br>Heavy casing design with liner and tie-back system was finalized to maximize accessibility and achieve proper isolation behind casing. A special pre-perforated liner was also designed to eliminate the pressure drop across the wellbore. The MRC drain was divided mainly into two sections, blank pipe and pre-perforated liner equipped with swell ...</code>	<code>The Clair field is the largest discovered oilfield on the UK continental shelf (UKCS) but has high reservoir uncertainty associated with a complex natural fracture network. The field area covers over 200 sq km with an estimated STOIIP of 7 billion barrels. The scale and complexity of the reservoir has led to a phased multi-platform development.<br>Phase 1 started production in 2005 with 20 wells drilled prior to an extended drill break. Five new wells (A21 to A25) were drilled and brought online during 2016/17 which increased platform production by c.70%. The new wells incorporated historic lessons to mitigate the risk of wellbore instability in the overburden and be robust to the dynamic uncertainties of the fractured reservoir. Many of the well outcomes and risk events were predicted and mitigated effectively, however the new wells still provided some surprises.<br>This paper presents a summary of the lessons from the historic Clair development wells which underpinned the recent drilling c...</code>

• Loss: <code>MultipleNegativesRankingLoss</code> with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Evaluation Dataset

offshore_energy_v1

• Dataset: offshore_energy_v1 at 4e9339c
• Size: 6,739 evaluation samples
• Columns: <code>anchor</code>, <code>positive</code>, and <code>negative</code>
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	<ul><li>min: 11 tokens</li><li>mean: 23.56 tokens</li><li>max: 52 tokens</li></ul>	<ul><li>min: 55 tokens</li><li>mean: 386.01 tokens</li><li>max: 1082 tokens</li></ul>	<ul><li>min: 45 tokens</li><li>mean: 382.6 tokens</li><li>max: 1175 tokens</li></ul>

• Samples:

  anchor	positive	negative
  <code>What is the importance of quantifying carbon emissions during cementing operations in decarbonization?</code>	<code>An important step in decarbonization is using an end-to-end approach to quantify carbon emissions during cementing operations. By careful analysis of the entire cementing operations process, it is then possible to measure and compare carbon emissions at various stages of the operation. Understanding and isolating the main drivers of the carbon emissions footprint enables making better choices and developing best alternatives with lower environmental impact.<br>The methodology considers the lifecycle assessment of cement from quarry extraction to well abandonment, and includes steps such as manufacturing of raw materials, transportation and logistics, and operations in the field. For these stages, careful quantification of emissions is performed based on the manufacturer's carbon emissions of cementing products, transportation (distance and means) to the bulk plant and rig site, and equipment-related emissions such as blending and pumping units. In some cases, when assessing the footprint ...</code>	<code>Objectives/Scope<br>There are many different views on the Energy Transition. What is agreed is that to achieve current climate change targets, the journey to deep decarbonisation must start now. Scope 3 emissions are clearly the major contributor to total emissions and must be actively reduced. However, if Oil and Gas extraction is to be continued, then operators must understand, measure, and reduce Scope 1 and 2 emissions. This paper examines the constituent parts of typical Scope 1 emissions for O&G assets and discusses a credible pathway and initial steps towards decarbonisation of operations.<br>Methods, Procedures, Process<br>Emissions from typical assets are investigated: data is examined to determine the overall and individual contributions of Scope 1 emissions. A three tiered approach to emissions savings is presented:<br>–<br>Reduce overall energy usage<br>–<br>Seek to Remove environmental losses<br>–<br>Replace energy supply with low carbon alternatives<br>A simple method, used to assess carbon emissions,...</code>
  <code>What factors must engineers consider during the drilling design phase?</code>	<code>The drilling of oil and gas wells involves several stages including the exploration phase, drilling design, and perforation techniques. In the exploration phase, geologists use seismic surveys to identify potential drilling locations. During the drilling design phase, engineers must consider factors such as wellbore stability, fluid mechanics, and formation pressures. Once the well is drilled, perforation techniques are applied to enhance the flow of hydrocarbons into the wellbore. The effectiveness of these techniques can significantly impact production rates and overall project success.</code>	<code>The extraction of crude oil and natural gas is typically carried out through drilling. Drilling uses different techniques to reach the petroleum reservoirs located deep underground. One key method is rotary drilling, where a drill bit is rotated while cutting through the earth's layers to create a wellbore. Rotary drilling is favored for its efficiency in penetrating hard rock layers. Another method is directional drilling, which allows operators to drill at various angles to reach reservoirs that are not directly beneath the drilling platform. This technique increases the area covered by the well and can optimize production. In addition, hydraulic fracturing enhances recovery rates by injecting fluids under high pressure to create fractures in the rock, increasing the permeability and allowing oil and gas to flow more freely. Lastly, the safety and environmental impacts of drilling techniques are a growing concern, and advancements are continually being sought to mitigate these effect...</code>
  <code>How does the 'Dissolved pore network' concept enhance matrix permeability in the modeling of carbonate oil reservoirs?</code>	<code>In this paper, we present a case study of using dual porosity dual permeability (DPDP) simulation for an offshore Abu Dhabi carbonate oil reservoir exhibiting complex flow behavior through matrix, fracture system and conductive faults. The main objective of the study is to present and explain the reservoir flow behaviors by constructing and using advanced reservoir geologic and simulation models. The results of the study will be utilized as part of the inputs for full field development plan.<br>Initially, an extensive work on the faults and fractures characterization was conducted to properly integrate this information into a dynamic model using DPDP modeling approach. However, the poor response of some wells or field sectors indicated the insufficiency of this concept to capture the full complexity of the reservoir system. Consequently, a new geological concept was proposed to represent the effect of enhanced matrix permeability related to facies dissolution process in the reservoir mode...</code>	<code>Integration of pressure-derived permeability thickness with other geological data plays a crucial role in estimating the apparent reservoir permeability, which is a key reservoir property required for reliable reservoir characterization as it governs fluid flow and greatly impacts decisions related to production, field development, and reservoir management. The geological model provides a representation of the subsurface reservoir, capturing the spatial distribution of lithology, porosity, permeability, and other geological properties. Analysis of pressure data provides valuable information on well condition, reservoir extent, and dynamic reservoir parameters. Integrating such data with the geological model is an enabler to better quantify and manage the uncertainty in the spatial 3D distribution of permeability away from well control.<br>This work proposes a methodology to build high-resolution geological models based on the available dynamic data, seismic data, and geologic interpretati...</code>

• Loss: <code>MultipleNegativesRankingLoss</code> with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• learning_rate: 2e-05
• num_train_epochs: 1
• warmup_ratio: 0.1

All Hyperparameters

<details><summary>Click to expand</summary>

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

</details>

Training Logs

Framework Versions

• Python: 3.10.12
• Sentence Transformers: 5.1.0
• Transformers: 4.53.3
• PyTorch: 2.8.0+cu128
• Accelerate: 1.9.0
• Datasets: 4.0.0
• Tokenizers: 0.21.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

<!-- ## Glossary *Clearly define terms in order to be accessible across audiences.* --> <!-- ## Model Card Authors *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.* --> <!-- ## Model Card Contact *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.* -->