finance-llama-nemotron-embed-1b-v2

SentenceTransformer based on nvidia/llama-nemotron-embed-1b-v2

This is a sentence-transformers model finetuned from nvidia/llama-nemotron-embed-1b-v2 on the financial-filings-sparse-retrieval-training dataset. It maps sentences & paragraphs to a 2048-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: nvidia/llama-nemotron-embed-1b-v2 <!-- at revision 113abe4acafa848e77ead9c0623205e511932348 -->
• Maximum Sequence Length: 2048 tokens
• Output Dimensionality: 2048 dimensions
• Similarity Function: Cosine Similarity
• Supported Modality: Text
• Training Dataset:
  • financial-filings-sparse-retrieval-training

<!-- - **Language:** Unknown --> <!-- - **License:** Unknown -->

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'LlamaBidirectionalModel'})
  (1): Pooling({'embedding_dimension': 2048, 'pooling_mode': 'mean', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("abanfalvi/finance-llama-nemotron-embed-1b-v2")
# Run inference
queries = [
    'deferred tax assets opening balance inventory loss 2022',
]
documents = [
    'The movements of deferred tax assets and deferred tax liabilities were as follows:\n\nFor the year ended December 31, 2022\n\nd. Deferred tax assets and liabilities\n\nDecember 31 2022 2021 Current tax liabilities Income tax payable $ 473,781 $ 255,744\n\nb. Income tax recognized in other comprehensive income\n\nOpening Balance Recognized in Profit or Loss Recognized in Other Comprehensive Income Closing Balance Deferred tax assets Temporary differences Unrealized loss on write-down of inventories $ 250 $ 8 $ - $ 258 Unrealized employee compensation 241 (241) - - Exchanges difference on foreign operations 73,357 - (23,271) 50,086 Unrealized exchange losses 15,010 (15,010) - - Total $ 88,858 $ (15,243) $ (23,271) $ 50,344 Deferred tax liabilities Temporary differences Unrealized exchange gain $ - $ 2,525 $ - $ 2,525\n\nFor the Year Ended December 31 2022 2021 Deferred tax In respect of the current year $ (23,271) $ (9,510) Translation of foreign operations\n\nc. Current tax assets and liabilities',
    'e. Information on unused loss carryforwards\n\nThe movements of deferred tax assets and deferred tax liabilities were as follows:\n\nFor the year ended December 31, 2021\n\nDecember 31 2022 Loss carryforwards Expiry in 2027 $ - Expiry in 2028 $ 46,056 Expiry in 2029 $ 72,486 Expiry in 2030 $ 97,191 Total $ 215,733\n\nOpening Balance Recognized in Profit or Loss Closing Balance Temporary differences Unrealized exchange gain and loss $ 3 $ (332) Unrealized inventory loss $ 1,113 $ 560 Others $ 402 $ (228) Total $ 1,518 $ -\n\nOpening Balance Recognized in Profit or Loss Closing Balance Temporary differences Unrealized exchange gain and loss $ 113 $ (110) Unrealized inventory loss $ 1,849 $ (736) Others $ (444) $ 846 Total $ 1,518 $ -\n\nUnused Amount Expiry Year $ 46,056 2028 $ 72,486 2029 $ 97,191 2030 Total $ 215,733\n\nd. Items for which no deferred tax assets have been recognized\n\nThe Company offset certain deferred tax assets and deferred tax liabilities which met the offset criteria.\n\nc. Deferred tax assets and liabilities\n\nLoss carryforwards as of December 31, 2022 comprised:\n\nFor the year ended December 31, 2022',
    'PILGRIM FOODSERVICE LIMITED\nNOTES TO THE FINANCIAL STATEMENTS (CONTINUED)\nFOR THE YEAR ENDED 30 APRIL 2025\n\n|   |   | 2025 £ | 2024 £ |\n|---|---|---|---|\n| 17 | Stocks |   |   |\n|   | Finished goods and goods for resale | 4,149,563 | 3,602,784 |\n\n|   |   | 2025 £ | 2024 £ |\n|---|---|---|---|\n| 18 | Debtors |   |   |\n|   | Amounts falling due within one year: |   |   |\n|   | Trade debtors | 4,056,601 | 3,939,861 |\n|   | Corporation tax recoverable | 386,649 | - |\n|   | Other debtors | 51,029 | 285,195 |\n|   | Prepayments and accrued income | 894,513 | 810,422 |\n|   |   | 5,388,792 | 5,035,478 |\n\n|   |   | Notes | 2025 £ | 2024 £ |\n|---|---|---|---|---|\n| 19 | Creditors: amounts falling due within one year |   |   |   |\n|   | Bank loans and overdrafts | 21 | 750,004 | 1,572,186 |\n|   | Obligations under finance leases | 22 | 929,382 | 221,527 |\n|   | Trade creditors |   | 6,972,922 | 5,358,604 |\n|   | Amounts due to group undertakings |   | 49,502 | - |\n|   | Corporation tax |   | 324,698 | 342,382 |\n|   | Other taxation and social security |   | 843,069 | 292,562 |\n|   | Other creditors |   | 3,361,115 | 802,545 |\n|   | Accruals |   |   | 2,269,019 |\n|   |   |   | 13,230,692 | 10,858,915 |\n\n|   |   | Notes | 2025 £ | 2024 £ |\n|---|---|---|---|---|\n| 20 | Creditors: amounts falling due after more than one year |   |   |   |\n|   | Bank loans and overdrafts | 21 | 1,174,972 | 1,925,178 |\n|   | Obligations under finance leases | 22 | 3,026,050 | 927,331 |\n|   | Shareholder loans | 21 | 1,925,125 | 681,083 |\n|   |   |   | 6,126,147 | 3,533,592 |\n\n-26-',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 2048] [3, 2048]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.5039, 0.4746, 0.2891]], dtype=torch.bfloat16)

<!-- ### Direct Usage (Transformers) <details><summary>Click to see the direct usage in Transformers</summary> </details> --> <!-- ### Downstream Usage (Sentence Transformers) You can finetune this model on your own dataset. <details><summary>Click to expand</summary> </details> --> <!-- ### Out-of-Scope Use *List how the model may foreseeably be misused and address what users ought not to do with the model.* -->

Evaluation

Metrics

Information Retrieval

• Dataset: financial_corpus
• Evaluated with <code>InformationRetrievalEvaluator</code>

<!-- ## Bias, Risks and Limitations *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* --> <!-- ### Recommendations *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* -->

Training Details

Training Dataset

financial-filings-sparse-retrieval-training

• Dataset: financial-filings-sparse-retrieval-training

• Size: 15,509 training samples

• Columns: <code>anchor</code>, <code>positive</code>, and <code>negative</code>

• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	<ul><li>min: 5 tokens</li><li>mean: 16.07 tokens</li><li>max: 61 tokens</li></ul>	<ul><li>min: 52 tokens</li><li>mean: 341.78 tokens</li><li>max: 1566 tokens</li></ul>	<ul><li>min: 56 tokens</li><li>mean: 373.04 tokens</li><li>max: 1713 tokens</li></ul>

• Samples:

  anchor	positive	negative
  <code>What is the unit of measurement used for Oil EUR on the vertical axis of the Relative Peer Well Performance chart?</code>	<code>Relative Peer Well Performance<br><br>Source: RSEG (2019)<br><br>Note: Bubbles are sized by well count and colored by oil %<br><br>Oil EUR (Mbbl/1,000')<br><br>| | 0 | 5 | 10 | 15 | 20 | 25 | 30 | 35 | 40 | 45 | 50 | |---|---|---|---|---|---|---|---|---|---|---|---|---| | Average Horizontal Interwell Spacing (ft) | 0 | 100 | 200 | 300 | 400 | 500 | 600 | 700 | 80|<br><br>Wellhead Liquids (%)<br><br><40 | 40-55 | >55</code>	<code>Exhibit B

  Total Stockholder Return for the Company and each of the Peer Companies shall be calculated in accordance with the following formula, with the result expressed as a percentage:

  The Company’s Relative Total Stockholder Return measured against the Peer Group shall be determined by first ranking the Company and each of the Peer Companies by their respective Total Stockholder Returns (highest to lowest) over the Performance Period. The Company’s Relative Total Stockholder Return shall be the Company’s percentile ranking determined from such numerical ranking, which percentile ranking shall be calculated as 100 multiplied by a fraction, the numerator of which is (x) the number of Peer Companies that are ranked lower than the Company by their respective Total Stockholder Returns and the denominator of which is (y) the number of Peer Companies in the Peer Group at the time of the determination minus one (1).

  The number of Conditional PSUs with which you are credited, if any, at t...</code> | | <code>Q3 FY22 EBITDA decline reasons inflation</code> | <code>Net sales is growing at healthy CAGR of 8.7% over 2 years, despite pandemic induced setbacks on our summer season brand sales<br><br>Metric YoY Gr 2 Year CAGR Net Sales 2.3 % 8.7% EBIDTA (34.8) % (7.0) % PBT * 1205.7 % 104.0% PAT * 1239.1 % 134.4%<br><br>Quarterly Performance - Q3 FY 22<br><br>EBIDTA de-grew mainly due to impact on gross margins on account of product mix and inflationary price increases in key raw materials & packing materials<br><br>PBT after exceptional items is growing mainly due to reduction in finance cost due to repayment of intercompany loan and nil exceptional item in the current quarter compared to Rs.342 million in previous year comparable period<br><br>The previous year comparable period includes exceptional items of Rs.342 million.</code> | <code>PVC Pipes and Fittings volume registered a y-o-y growth of 4.2% to 1,58,266 MT.

  Business continues to be net debt free

  PAT grew by 30% from Rs 431 Cr to 560 Cr

  EBITDA dropped to Rs 242 Cr (vs. Rs. 346 Cr)

  Segment Revenue – Q3 FY22

  All Numbers reported in the presentation are on standalone basis

  Segment Volume – Q3 FY22

  Total revenue registered a y-o-y growth of 38% to Rs. 3,054 Cr

  Segment MT PVC P&F 46,994 PVC Resin 43,464

  PVC Resin volume registered a y-o-y decline of 36% to 43,464 MT

  PVC Pipes & Fittings volume registered a y-o-y decline of 15.3% to 46,994 MT.

  PVC Resin volume registered a y-o-y decline of 9.4% to 1,45,742 MT

  Total revenue registered a y-o-y decline of 5.7% to Rs. 1,005 Cr from Rs.1066 Cr

  Strong liquidity and healthy balance sheet

  Profitability slightly impacted by higher input prices

  PAT decreased by 31% from Rs 256 Cr to Rs 178 Cr

  Q3 FY22 9M FY22

  Segment Rs Cr PVC Resin 832 P&F 636

  Strong business performance despite lower demand

  Strong YTD perfo...</code> | | <code>What was the total value of trade creditors in 2022?</code> | <code>2022 (£) 2021 (£) Trade debtors 1,849,235 1,877,661 Other debtors 105,874 75,543 Prepayments and accrued income 36,698 61,917 Total 1,991,807 2,015,121 Deferred tax asset (note 10) 341,000 408,000 Total Debtors 2,332,807 2,423,121

  The following are the major deferred tax liabilities and assets recognised by the company and movements thereon:

  The deferred tax falling due after more than one year relates to losses carried forward.

  Amounts falling due within one year:

  10-

  Movements in the year:

  Assets 2022 (£) Assets 2021 (£) Accelerated capital allowances (5,000) (6,000) Tax losses 346,000 414,000 Total Balances 341,000 408,000

  Balances:

  2022 (£) 2021 (£) Trade creditors 160,773 61,453 Amounts owed to group undertakings 732,753 1,370,540 Taxation and social security 234,304 239,278 Other creditors 284,740 460,640 Accruals and deferred income 903,386 1,042,016 Total Creditors 2,315,956 3,173,927

  9 Creditors: amounts falling due within one year

  8 Debtors

  10 Deferred taxation

  20...</code> | <code>15 Other non-current liabilities

  2024 (£m) 2023 (£m) Trade and other debtors 22 22 Prepayments and accrued income 12 12 Total 34 34

  The decrease in provisions for impairment of trade debtors and accrued income of £18m (2022/23: £18m decrease) is equal to the credit to the consolidated income statement of £14m (2022/23: £11m credit), and write-offs of trade debtors of £4m (2022/23: £7m).

  13 Debtors

  2024 (£m) 2023 (£m) Lease liabilities 121 120 Deferred income¹ - 25 Total 121 145

  The Directors consider that the carrying amount of trade and other debtors is approximate to their fair value. Further details about the Group's credit risk management practices are disclosed in Note 16.

  14 Creditors

  The credit to the consolidated income statement for the year in relation to the release of impairment of trade debtors and accrued income was £14m (2022/23: £11m credit), as disclosed in Note 3.

  Trade and other debtors are shown after deducting a provision for impairment against tenant debto...</code> |

• Loss: <code>MultipleNegativesRankingLoss</code> with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false,
      "directions": [
          "query_to_doc"
      ],
      "partition_mode": "joint",
      "hardness_mode": null,
      "hardness_strength": 0.0
  }
  

Evaluation Dataset

financial-filings-sparse-retrieval-training

• Dataset: financial-filings-sparse-retrieval-training

• Size: 2,738 evaluation samples

• Columns: <code>anchor</code>, <code>positive</code>, and <code>negative</code>

• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	<ul><li>min: 4 tokens</li><li>mean: 16.07 tokens</li><li>max: 52 tokens</li></ul>	<ul><li>min: 56 tokens</li><li>mean: 325.18 tokens</li><li>max: 1272 tokens</li></ul>	<ul><li>min: 64 tokens</li><li>mean: 355.84 tokens</li><li>max: 1099 tokens</li></ul>

• Samples:

  anchor	positive	negative
  <code>What was the regional distribution of holdings within the United States and Canada during the spring of 2020?</code>	<code>OWNERSHIP STAKE<br><br>March 2020 March 2019 Fully operational 97% 97% Construction 3% 3%<br><br>70 HICL ANNUAL REPORT 2020<br><br>March 2020 March 2019 UK 76% 77% EU 17% 15% North America 7% 8%<br><br>March 2020 March 2019 PPP projects 72% 71% Demand-based assets 20% 21% Regulated assets 8% 8%<br><br>MARKET SEGMENT<br><br>GEOGRAPHIC LOCATION<br><br>INVESTMENT STATUS<br><br>3.6 Portfolio Analysis<br><br>March 2020 March 2019 100% ownership 26% 25% 50%-100% ownership 34% 32% Less than 50% ownership 40% 43%<br><br>SECTOR<br><br>as at 31 March 2020<br><br>March 2020 March 2019 Accommodation 11% 11% Education 14% 15% Electricity, Gas & Water 8% 8% Health 30% 28% Fire, Law & Order 7% 7% Transport 30% 31%</code>	<code>PROPERTY, PLANT & EQUIPMENT

  (a) At cost, except Factory land which is at cost, less amounts written off.

  Particulars Land & Building Factory Building Office Equipment's Furniture & Fixtures Plant & Machinery Vehicles Computers Electrical fittings Total Tangible Assets Intangible Assets Software Total Intangible Assets GROSS BLOCK As on 1st April 2019 17.22 443.21 46.92 37.11 2,339.36 8.32 25.50 1.80 2,919.44 4.67 4.67 Additions - 3.74 0.32 45.17 - 1.53 0.40 51.16 3.76 3.76 Disposals - - - 2.64 - - - 2.64 - - At 31 March 2020 17.22 443.21 50.67 37.43 2,381.89 8.32 27.03 2.20 2,967.96 8.43 8.43 As on 1st April 2020 17.22 443.21 50.67 37.43 2,381.89 8.32 27.03 2.20 2,967.96 8.43 8.43 Additions 24.52 - 49.09 42.36 2.08 6.27 124.33 - - Disposals - - - - - - - - - - - At 31 March 2021 41.74 443.21 100.67 80.59 2,430.98 50.68 29.11 8.47 3,092.29 8.43 8.43 DEPRECIATION As on 1st April 2019 - 320.72 41.98 27.39 2,153.01 5.39 19.32 0.12 2,567.94 4.67 4.67 Charge for the year 17.21 2.98 2.52 68...</code> | | <code>What is the interest rate charged on the loan to Karbon Carlton Staindrop LLP?</code> | <code>Unrelieved tax losses of £68,994 (2023: £68,994) remain available to offset against future taxable profits.

  12

  2024 (£) 2023 (£) Other debtors 1,821 875 Taxation 42,871 - Work in progress 8,928 - Amounts owed by Karbon Carlton Staindrop LLP 207,665 875 261,285

  On the loan to Karbon Carlton Staindrop LLP, interest is charged at 7% per annum.

  7 TAXATION (CONTINUED)

  The tax assessed is lower than the standard rate of corporation tax in the UK (19%). The differences are explained below.

  8 DEBTORS: amounts due within one year

  9 DEBTORS: amounts due after one year

  Current tax reconciliation

  2024 (£) 2023 (£) Profit on ordinary activities before tax 5,682 374,965 Tax on profit on ordinary activities at the standard rate of corporation tax of 25% (2023: 19%) 1,421 71,243 Effects of: Expenses not deductible for tax purposes 3,113 6,157 Group relief claimed (4,534) (77,400) Current tax charge for period

  2024 (£) 2023 (£) Amounts owed by Karbon Carlton Staindrop LLP - loan capital 1,776...</code> | <code>RWK GOODMAN LLP NOTES TO THE FINANCIAL STATEMENTS (CONTINUED) FOR THE YEAR ENDED 31 MARCH 2025

  18 Borrowings Group and LLP

  	2025	2024
  Bank loan	7,341,667	6,441,667
  Bank overdrafts	5,359,258	4,995,132
  	12,700,925	11,436,799
  Payable within one year	12,459,258	11,095,132
  Payable within two to five years	241,667	341,667

  The bank loan and overdrafts are secured by a fixed charge over book and other debts and a floating charge over all assets. A revolving loan facility of £8,000,000 was agreed on 7 January 2025, replacing the existing RCF agreement dated 25 January 2022. The loan is repayable in full at the end of the term on 7 January 2030. Interest is charged quarterly at a rate of 2% over base rate. A bank loan of £500,000 was agreed with HSBC UK Bank pic in August 2023. The loan is repayable over 60 months by monthly instalments of £9,903. Interest is charged at a rate of 2% over base rate.

  **19 Provision...</code> | | <code>Newmont Corporation 2022 restricted cash reconciliation</code> | <code>```markdown NEWMONT CORPORATION CONSOLIDATED STATEMENTS OF CASH FLOWS

  		Year Ended December 31,
  	2022	2021	2020
  		(in millions)

  Reconciliation of cash, cash equivalents and restricted cash:
  Cash and cash equivalents	$ 2,877	$ 4,992	$ 5,540
  Restricted cash included in Other current assets	1	2	2
  Restricted cash included in Other non-current assets	66	99	106
  Total cash, cash equivalents and restricted cash	$ 2,944	$ 5,093	$ 5,648

  Supplemental cash flow information:
  Income and mining taxes paid, net of refunds	$ 1,122	$ 1,534	$ 400
  Interest paid, net of amounts capitalized	$ 172	$ 229	$ 261

  (1) Acquisitions, net for the year ended December 31, 2021 is primarily related to the asset acquisition of the remaining 85.1% of GT Gold. Refer to Note 1 for additional information. The accompanying ...</code> | <code>Component Percentage Personal Bonus 6% Base Salary 14% Company Bonus 15% Restricted Stock Units 20% Performance Stock Units 45%<br><br>Component Objectives/Alignment Stock Price Performance • Value varies with NEM performance • Retention component Relative Stock Performance Long-term incentive to outperform gold competitors: • Absolute share price performance • Relative TSR performance Operating Performance Growth Pipeline/ Sustainability • Safety, Free Cash Flow, CSC, ROCE • Reserves and Resources • Sustainability and External Relations Leadership Measures • Individual objectives (with defined targets) • Leadership Pipeline results Base Salary • Adjusted for performance, scope • Market rate<br><br>Objectives/Alignment<br><br>Incentive vehicles balance key performance elements<br><br>Pay mix (CEO % shown)<br><br>Executive Compensation Structure<br><br>BANK OF AMERICA 2020 GLOBAL METALS, MINING & STEEL CONFERENCE NEWMONT CORPORATION<br><br>Plans support value chain to operating and market performance</code> |

• Loss: <code>MultipleNegativesRankingLoss</code> with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false,
      "directions": [
          "query_to_doc"
      ],
      "partition_mode": "joint",
      "hardness_mode": null,
      "hardness_strength": 0.0
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 2
• max_steps: 100
• learning_rate: 3e-05
• lr_scheduler_type: constant_with_warmup
• warmup_steps: 0.03
• gradient_accumulation_steps: 8
• fp16: True
• gradient_checkpointing: True
• load_best_model_at_end: True

All Hyperparameters

<details><summary>Click to expand</summary>

• per_device_train_batch_size: 2
• num_train_epochs: 3.0
• max_steps: 100
• learning_rate: 3e-05
• lr_scheduler_type: constant_with_warmup
• lr_scheduler_kwargs: None
• warmup_steps: 0.03
• optim: adamw_torch_fused
• optim_args: None
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• optim_target_modules: None
• gradient_accumulation_steps: 8
• average_tokens_across_devices: True
• max_grad_norm: 1.0
• label_smoothing_factor: 0.0
• bf16: False
• fp16: True
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• gradient_checkpointing: True
• gradient_checkpointing_kwargs: None
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• use_liger_kernel: False
• liger_kernel_config: None
• use_cache: False
• neftune_noise_alpha: None
• torch_empty_cache_steps: None
• auto_find_batch_size: False
• log_on_each_node: True
• logging_nan_inf_filter: True
• include_num_input_tokens_seen: no
• log_level: passive
• log_level_replica: warning
• disable_tqdm: False
• project: huggingface
• trackio_space_id: trackio
• per_device_eval_batch_size: 8
• prediction_loss_only: True
• eval_on_start: False
• eval_do_concat_batches: True
• eval_use_gather_object: False
• eval_accumulation_steps: None
• include_for_metrics: []
• batch_eval_metrics: False
• save_only_model: False
• save_on_each_node: False
• enable_jit_checkpoint: False
• push_to_hub: False
• hub_private_repo: None
• hub_model_id: None
• hub_strategy: every_save
• hub_always_push: False
• hub_revision: None
• load_best_model_at_end: True
• ignore_data_skip: False
• restore_callback_states_from_checkpoint: False
• full_determinism: False
• seed: 42
• data_seed: None
• use_cpu: False
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• dataloader_prefetch_factor: None
• remove_unused_columns: True
• label_names: None
• train_sampling_strategy: random
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• ddp_backend: None
• ddp_timeout: 1800
• fsdp: []
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• deepspeed: None
• debug: []
• skip_memory_metrics: True
• do_predict: False
• resume_from_checkpoint: None
• warmup_ratio: None
• local_rank: -1
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

</details>

Training Logs

<details><summary>Click to expand</summary>

• The bold row denotes the saved checkpoint.

</details>

Training Time

• Training: 4.6 hours

Framework Versions

• Python: 3.12.13
• Sentence Transformers: 5.4.1
• Transformers: 5.5.0
• PyTorch: 2.10.0+cu128
• Accelerate: 1.13.0
• Datasets: 4.3.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{oord2019representationlearningcontrastivepredictive,
      title={Representation Learning with Contrastive Predictive Coding},
      author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
      year={2019},
      eprint={1807.03748},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/1807.03748},
}

<!-- ## Glossary *Clearly define terms in order to be accessible across audiences.* --> <!-- ## Model Card Authors *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.* --> <!-- ## Model Card Contact *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.* -->