econo-sentence-v2

SentenceTransformer based on samchain/EconoBert

This is a sentence-transformers model finetuned from samchain/EconoBert. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. This model is the second version of the previous samchain/econo-sentence-v1. They both used the same base model but have been trained on different datasets.

The model has been evaluated both on the newest dataset and its precedent version:

• econo-pairs-v2 is the newest dataset. Check its card here
• econo-pairs-v1is the oldest version. Check its card here

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: samchain/EconoBert <!-- at revision 1554ddcdc25e1886cc43e05b9c77a2b8c4888da6 -->
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

<!-- - **Training Dataset:** Unknown --> <!-- - **Language:** Unknown --> <!-- - **License:** Unknown -->

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("samchain/econosentence-v2")
# Run inference
sentences = [
    "education shocks arising from the pandemic could signify the largest reversal in human development on record, equivalent to erasing all the progress in human development of the past six years4. the risk of reversing progress in achieving financial inclusion is accentuating why financial inclusion matters now more than ever. optimising islamic finance for inclusive economic recovery policy and regulatory responses across the globe, and likewise in malaysia, have been focused on safeguarding economic resilience, managing risks to financial stability and minimising repercussions to society. this is done in tandem with the different stages of the pandemic – namely, containment, stabilisation and recovery. at the onset of the crisis, governments along with financial regulators have deployed sizeable stimulus packages and various assistance programmes in order to contain the crisis and stabilise the economy. this includes addressing demand and supply disruptions, maintaining cash flows and keeping workers employed. in malaysia, the total stimulus package amounted to usd 73. 55 billion ( rm3056 billion ) with an additional fiscal injection by the government totalling 1 / 4 bis central bankers'speeches rm45 billion. as at september 2020, a total of 2. 63 million workers and 321, 000 employers had benefitted from the wage subsidy programme, involving an expenditure of rm10. 4 billion. to provide further stimulus to the economy, the bank has reduced the overnight policy rate ( opr ) by a cumulative 125 basis points ( from 3. 00 % to 1. 75 % ) this year, alongside reduction in the statutory reserve requirement by 100 basis points ( from 3. 00 % to 2. 00 % ). the reduction in the opr is intended to provide additional policy stimulus to accelerate the pace of economic recovery. the financial industry, including islamic financial institutions, also lent support to their borrowers and customers. in the first half of 2020, a total of rm120 billion7 was disbursed in lending / financing to smes, with more accounts being approved8 in aggregate in 2020 compared to the same period in previous years. islamic financial institutions and related associations have been actively educating and reaching out to affected borrowers about the financial assistance programmes available in response to the pandemic. the takaful and insurance industry also facilitated affected certificate holders by offering temporary deferment of contribution and premium to promote continuity of takaful protection coverage. more than 1. 1 million9 certificate and policyholders have benefited from this relief measure. while the",
    'education at an early age. a credit union is similar to a commercial bank, it can provide the same products and services like a commercial bank, but what is different is that the credit union is owned by members who are the shareholders and at the same time are customers to the credit union. credit unions can provide different saving accounts to suit the needs of their members. they can create saving products such as junior saving accounts for the children. credit unions provide lending products for their members. and credit unions can provide insurance coverage for their members as well. one of the reasons people explain why they don ’ t want to open an account with a commercial bank is that because banks charge fees to keep their accounts. while paying fees is something that we all cannot avoid because provision of banking services costs money, in a credit union the interests, fees and charges that are paid by members is distributed back to the members as net interest income. i say this because i am also member of our staff credit union called bokolo credit union limited day one when i join the central bank of solomon islands and know the benefit credit unions can give members. as the registrar for credit unions, i take note of the progress in the growth of the assets of both the soltuna employees credit union limited and the tuna trust credit union limited. if the board and management of these credit unions continue to manage these credit unions professionally, they will both be the vehicle to encourage our children and youth here in noro and munda community to learn how to save and earn and become good financial and economic citizens of our country. finally, let me take this opportunity to thank the many people who made this global money week celebration possible. to the board of directors of both tuna trust credit union limited and soltuna employees credit union limited. i salute you for accepting our request to work together to host this global money week. i would like to thank selwyn talasasa, one of the long serving credit union promoters who has been a credit union person since day one of his career. he has been very helpful in organizing our global money week. thank you selwyn. i also thank the global money week committee for their assistance in organising this global money week. and thanks to the management of both national fisheries development limited and soltuna limited for your support in hosting this global money week. thank you nfd and soltuna, you made us all proud as good cooperate citizens. to our noro town clerk for your support.',
    'fund ( uncdf ) are also actively developing financial inclusion knowledge, and supporting countries in implementing financial inclusion strategies globally. over the years, we have since witnessed the rapid deployment of innovative financial inclusion solutions that have changed the lives of many. since the introduction of the mpesa mobile banking service in kenya, the number of adults with access to financial services has increased from 42 % in 2011 to 75 % today. agent banking in malaysia has provided consumers with an innovative and alternative channel to access financial services, resulting in the percentage of sub - districts served by financial services access point to increase from 46 % in 2011 to 96 % in 2014. this has enabled 99 % of malaysians, particularly those in rural areas, to conveniently access and benefit from financial services. the microfinance “ global financial development report 2014 ”, world bank, 2014. discussion note “ redistribution, inequality, and growth ”, imf, 2014. bis central bankers ’ speeches institution grameen bank in bangladesh has extended credit through its 2, 500 branches to almost 8 million people, with 60 % of them lifted from poverty. the loan recovery rate is higher than 98 percent. the experience in bangladesh, in particular, dispels the myth that the poor is not bankable. of proportionate regulation and the global standards the proportionate application of global standards for financial regulation is a critical factor in enabling innovative financial inclusion solutions, ensuring its delivery in a safe and sound manner. my remarks today will touch on the following topics : recent developments in global standards and financial inclusion ; focus areas to advance proportionality in practice ; and the importance of the global symposium to galvanise action. recent developments in global standards and financial inclusion global standards developed after the global financial crisis were designed to address the financial stability and integrity issues that came to the fore in developed countries. many of these issues involved the activities of global systemically important financial institutions. the effects of these standards on financial inclusion, such as the impact on smaller financial institutions in developing countries most likely were not taken into consideration. fortunately, standard - setting bodies have now recognised the principle of proportionality, emphasising on the balance between objectives of financial stability, integrity and inclusion. however, the real challenge lies in implementing proportionality in practice. if principles are not implemented in a proportionate manner, there could be unintended consequences for financial inclusion. for example, despite the financial action task force ’ s guidance on the implementation of a risk - based approach to anti money laundering /',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

<!-- ### Direct Usage (Transformers) <details><summary>Click to see the direct usage in Transformers</summary> </details> --> <!-- ### Downstream Usage (Sentence Transformers) You can finetune this model on your own dataset. <details><summary>Click to expand</summary> </details> --> <!-- ### Out-of-Scope Use *List how the model may foreseeably be misused and address what users ought not to do with the model.* -->

Evaluation

Metrics

Semantic Similarity

• Datasets: econo-pairs-v2 and econo-pairs-v1
• Evaluated with <code>EmbeddingSimilarityEvaluator</code>

Econo-sentence-v2 evaluation results:

Econo-sentence-v1 evaluation results:

The v2 and v1 differs in terms of training as the v2 has been train on 0.5 labels, while the v1 is only trained on 1 and 0. The results show that the v2 still performs on par with its predecessor on the prior dataset while improving the performance by 10 points on the latest.

<!-- ## Bias, Risks and Limitations *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* --> <!-- ### Recommendations *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* -->

Training Details

Training Dataset

Train set of econo-pairs-v2

• Size: 45,044 training samples
• Columns: <code>text1</code>, <code>text2</code>, and <code>label</code>
• Approximate statistics based on the first 1000 samples:

  	text1	text2	label
  type	string	string	float
  details	<ul><li>min: 3 tokens</li><li>mean: 454.6 tokens</li><li>max: 505 tokens</li></ul>	<ul><li>min: 6 tokens</li><li>mean: 458.88 tokens</li><li>max: 505 tokens</li></ul>	<ul><li>min: 0.0</li><li>mean: 0.49</li><li>max: 1.0</li></ul>

• Samples:

  text1	text2	label
  <code>in credit intermediation is not disrupted. i hope the analysis and observations i have shared today can assist with a process of learning, evolving, and improving that will prevent future disruptions. thank you. bis central bankers ’ speeches bis central bankers ’ speeches bis central bankers ’ speeches</code>	<code>richard byles : update and outlook for the jamaican economy monetary policy press statement by mr richard byles, governor of the bank of jamaica, at the quarterly monetary policy report press conference, kingston, 21 february 2024. * * * introduction good morning and welcome to our first quarterly monetary policy report press conference for 2024. the year has started with inflation being above the bank's inflation target. statin reported last week that headline inflation at january 2024 was 7. 4 per cent, which is higher than the outturns for the previous three months, and above the bank's target of 4 to 6 per cent. core inflation, however, which excludes food and fuel prices from the consumer price index, was 5. 9 per cent, which is lower than the 7. 1 per cent recorded in january 2023. the inflation outturn at january is higher than the bank had projected in november 2023. as we communicated then, in the wake of the announcement by the minister of finance and the public service of th...</code>	<code>0.0</code>
  <code>european central bank : press conference - introductory statement introductory statement by mr jean - claude trichet, president of the european central bank, frankfurt am main, 4 may 2006. * * * ladies and gentlemen, let me welcome you to our press conference and report on the outcome of today ’ s meeting of the ecb ’ s governing council. the meeting was also attended by commissioner almunia. on the basis of our regular economic and monetary analyses, we have decided to leave the key ecb interest rates unchanged. overall, the information which has become available since our last meeting broadly confirms our earlier assessment of the outlook for price developments and economic activity in the euro area, and that monetary and credit growth remains very dynamic. against this background, the governing council will exercise strong vigilance in order to ensure that risks to price stability over the medium term do not materialise. such vigilance is particularly warranted in a context of ample...</code>	<code>vitor constancio : strengthening european economic governance – surveillance of fiscal and macroeconomic imbalances speech by mr vitor constancio, vice - president of the european central bank, at the brussels economic forum, brussels, 18 may 2011. * * * thank you very much for the invitation to participate in this panel today. the session is intended to focus on “ surveillance of fiscal and macroeconomic imbalances ”, which arguably is the most important strand of the economic governance package. but one should consider that this package – beyond the 6 legislative acts – also contains three other strands : the euro plus pact, the creation of the european stability mechanism and the setting - up of the european system of financial supervision. a fundamental strengthening of economic governance in the euro area requires simultaneous progress in all three areas. the first strand is necessary to prevent and correct imbalances ; the second to ensure the conditions for future growth and com...</code>	<code>0.5</code>
  <code>. innovation is already altering the power source of motor vehicles, and much research is directed at reducing gasoline requirements. at present, gasoline consumption in the united states alone accounts for 11 percent of world oil production. moreover, new technologies to preserve existing conventional oil reserves and to stabilize oil prices will emerge in the years ahead. we will begin the transition to the next major sources of energy perhaps before midcentury as production from conventional oil reservoirs, according to central tendency scenarios of the energy information administration, is projected to peak. in fact, the development and application of new sources of energy, especially nonconventional oil, is already in train. nonetheless, it will take time. we, and the rest of the world, doubtless will have to live with the uncertainties of the oil markets for some time to come.</code>	<code>oil industry to build inventories, demand from investors who have accumulated large net long positions in distant oil futures and options is expanding once again. such speculative positions are claims against future oil holdings of oil firms. currently, strained capacity has limited the ability of oil producers to quickly satisfy this markedly increased demand for inventory. adding to the difficulties is the rising consumption of oil, especially in china and india, both of which are expanding economically in ways that are relatively energy intensive. even the recent notable pickup in opec output, by exhausting most of its remaining excess capacity, has only modestly satisfied overall demand. output from producers outside opec has also increased materially, but investment in new producing wells has lagged, limiting growth of production in the near term. crude oil prices are also being distorted by shortages of capacity to upgrade the higher sulphur content and heavier grades of crude oi...</code>	<code>1.0</code>

• Loss: <code>CoSENTLoss</code> with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }
  

Evaluation Dataset

Test set of econo-pairs-v2

• Size: 11,261 evaluation samples
• Columns: <code>text1</code>, <code>text2</code>, and <code>label</code>
• Approximate statistics based on the first 1000 samples:

  	text1	text2	label
  type	string	string	float
  details	<ul><li>min: 4 tokens</li><li>mean: 451.65 tokens</li><li>max: 505 tokens</li></ul>	<ul><li>min: 6 tokens</li><li>mean: 457.79 tokens</li><li>max: 505 tokens</li></ul>	<ul><li>min: 0.0</li><li>mean: 0.5</li><li>max: 1.0</li></ul>

• Samples:

  text1	text2	label
  <code>the drying up of the unsecured interbank market a price worth paying for lower interest rate volatility and tighter interest rate control in the money markets? at present, it is difficult to envisage how the unsecured interbank money market could be revived in full in the near future, reaching the pre - global financial crisis levels. this is due both to regulatory reasons and to fragmentation. post - crisis, banks appear reluctant to lend each other in the unsecured interbank market, pricing such lending at levels that are sufficient to compensate for counterparty credit risk. at the same time, short - term ( below 6 months ) interbank funding does not contribute towards satisfying the nsfr requirements. as a result, the unsecured segment of the interbank market is unattractive for prospective borrowers and is not expected to regain importance. furthermore, heterogeneity within and across the banking systems of euro area countries impedes the efficient redistribution of reserves acros...</code>	<code>##obalisation, artificial intelligence, as well as the green transition and climate change adaptation policies. 1 / 7 bis - central bankers'speeches given the uncertain overall impact of these effects and the significant unknowns and ambiguity surrounding the computation of natural interest rates, their prospective level and evolution are not easily determined. turning now to the lessons learnt, experience gained during the past crises has been extremely valuable. first, monetary policy needs to remain flexible so that it can accommodate potential future shocks to price stability, of any nature and direction. second, financial stability considerations need to be taken into account in the decision making of monetary policymakers, in their pursuit of price stability. the past decade provided us with new insights into what monetary policy can do. we are now better prepared to implement the appropriate monetary policy mix, with standard and non - standard tools, if we were to face new chal...</code>	<code>1.0</code>
  <code>therefore worries me that greece might go backwards in terms of its reform agenda. suffice to say, it is not enough to correct misalignments solely at the national level. we also need structural reforms at the european level. and here, a lot has been done since the crisis broke out. first, the rules of the stability and growth pact were tightened and a fiscal compact was adopted in order to restore confidence in public finances. second, a procedure for identifying macroeconomic imbalances at an early stage was established. and third, a crisis mechanism was set up to serve as a “ firewall ”, safeguarding the stability of the financial system in the euro area. in addition to these measures, the euro area took a major step toward deeper financial integration. on 4 november 2014, the first pillar of a european banking union was put in place. on that day, the ecb assumed responsibility for supervising the 123 largest banks in the euro area. together, the banks concerned account for more tha...</code>	<code>andreas dombret : the euro area – where do we stand? speech by dr andreas dombret, member of the executive board of the deutsche bundesbank, at the german - turkish chamber of trade and commerce, istanbul, 10 february 2015. * 1. * * introduction ladies and gentlemen thank you for inviting me to speak here at the german - turkish chamber of industry and commerce in istanbul. isaac newton once observed that “ we build too many walls and not enough bridges ”. istanbul has for centuries been a bridge between two continents – asia and europe. it seems to me that istanbul is therefore an excellent place to hold this year ’ s g20 meetings. after all, the purpose of these meetings is to build bridges between nations by fostering dialogue and the mutual exchange of information. and this objective is of course also pursued by the german - turkish chamber of trade and commerce. moreover, mutual exchange is essential in a globalised world where countries are no longer isolated islands but part of ...</code>	<code>1.0</code>
  <code>for vital issues regarding the agenda. i desire you all success with the materialization and accomplishment of your present and future projects and i conclude by wishing all the foreign guests at this conference a good stay in romania. i am now giving the floor to mr kurt puchinger, coordinator of the eu strategy for the danube region and chair of the first session, to start the conference. thank you. bis central bankers ’ speeches</code>	<code>national bank of romania, as they capture appropriately the opinions of the real sector on some key issues, such as : ( i ) the most pressing problems that firms are facing in their activity ; ( ii ) the investment needs and priorities ; ( iii ) challenges raised by climate change and energy efficiency. the survey shows that firms in romania have become less optimistic regarding investment conditions for the year ahead. while this is a worrying trend, we are rather positive that romania will be able to overcome the main problem identified, the energy costs, in the near future. i would add here that the government has implemented several measures in this respect, such as electricity and natural gas price capping schemes, the compensation of the price of motor fuel / liter and caps on the firewood price. also, we are aware that the war in ukraine and the related sanctions will continue to generate considerable uncertainties and risks to the outlook for economic activity, through possibly...</code>	<code>0.5</code>

• Loss: <code>CoSENTLoss</code> with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• learning_rate: 2e-05
• num_train_epochs: 2
• warmup_ratio: 0.1
• fp16: True
• batch_sampler: no_duplicates

All Hyperparameters

<details><summary>Click to expand</summary>

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 2
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional

</details>

Training Logs

Framework Versions

• Python: 3.10.12
• Sentence Transformers: 3.4.1
• Transformers: 4.49.0
• PyTorch: 2.1.0+cu118
• Accelerate: 1.4.0
• Datasets: 3.3.1
• Tokenizers: 0.21.0

Citation

Samuel Chaineau

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

CoSENTLoss

@online{kexuefm-8847,
    title={CoSENT: A more efficient sentence vector scheme than Sentence-BERT},
    author={Su Jianlin},
    year={2022},
    month={Jan},
    url={https://kexue.fm/archives/8847},
}

<!-- ## Glossary *Clearly define terms in order to be accessible across audiences.* --> <!-- ## Model Card Authors *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.* --> <!-- ## Model Card Contact *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.* -->