finetune_Qwen_3_.8B_Embedding

SentenceTransformer based on unsloth/Qwen3-Embedding-0.6B

This is a sentence-transformers model finetuned from unsloth/Qwen3-Embedding-0.6B on the train and test datasets. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: unsloth/Qwen3-Embedding-0.6B <!-- at revision f2fddb42505bde9feaf19f0967b01dce52e764c6 -->
• Maximum Sequence Length: 768 tokens
• Output Dimensionality: 1024 dimensions
• Similarity Function: Cosine Similarity
• Training Datasets:
  • train
  • test

<!-- - **Language:** Unknown --> <!-- - **License:** Unknown -->

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 768, 'do_lower_case': False, 'architecture': 'PeftModelForFeatureExtraction'})
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': True, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    "Given that 66 out of 79 TSS/SCP sites were encumbrance-free by August 31, 2016, what potential liabilities arise if the contractor argues that these specific encumbrances, despite their temporary nature, fundamentally altered the project's critical path and increased overall costs?",
    "To conclude and with reference to above sub-paras under 2.1.2, it is stated that out of 79 TSS/SCPs sites in MS-1 and MS-2 section, 66 TSS/SCPs sites were encumbrance free as on 31.08.2016. Regarding encumbrances on the remaining TSS/SCP sites, these were mostly issue of interface management with civil and track Contractor mainly for removal of their material which is temporarily stacked at site. However, even on these sites, which were available to the contractor and are encumbrance free as on 31.08.2016, the progress of work was nil for any of the CCP activities such as ( site survey, clearing and grubbing, site preparatory works, earthwork etc. which are preceding activities to construction activity ). (Engineer's letter dated 03.10.2017 & 02.07.2018 at SN 53 & 54,). As such from above. it is clear that non availability of the encumbrance free sites to the contractor is not a reason for contractor's non- commencement of the work and non-achievement of MS-1 & MS-2 or cause for any delay.The Contractor has not substantiated as to how non handing over of SCP sites or individual obstructions at these sites prevented him for proceeding with Permanent Work and affected / delayed the permanent work. The Contractor has not established the impact of the encumbered conditions to the time for completion and hence, it is clear that non availability of the encumbrance free sites to the contractor is not a reason for contractor's non- commencement of the work/ non-achievement of MS-1 & MS-2 or any cause of delay.In addition to above, Contractor's attention is invited to Engineer's letters on the subject of SLOW Progress of the Contractor letter cited at SN 55, SN56, SN57, SN58, SN59 (for delays accrued up to 30.06.2021), which gives details of item wise deficiency /shortfall in progress of various works of the SLT4 Contractor for reasons entirely attributable to the Contractor.",
    '“5.2 Functional and technical specifications 5.2.1 The comparison of functional and technical specifications "Annexure G" (EIRENE Functionality Comparison between Poland Network and STP-17 contract) has been provided which compares the requirement in India DFCC project and what was tested / deployed in Poland project. 5.2.2 As per WDFC contract, vendor has to deploy GSM-R network compliant with EIRENE FRS v7.4.0/SRSv15.4.0. 5.2.3 Moreover, it has been observed that the WDFC technical specification is based on EIRENE FRS/SRS. 76 C-060 77 C-059 ICC Case No. 28100/HTG EXPERT REPORT OF DR BOYD MURRAY PAGE 97 Both optional specification and mandatory specification are the same in Poland project and in STP-17 project. 5.3 Hardware & software levels of all Equipment 5.3.1 Based on document on "IOT Declaration/IOT Certificate related to Poland Network" (Annexure A, Annexure C) and Nokia proposal for WDFC2 network, it is understood that same hardware version will supplied by Nokia as is used in Poland Network. 5.3.2 The interoperability testing performed in Poland between KCC and Nokia have been evaluated between the KCC Core and Nokia Flexi BSS. (Annexure A, Annexure C) 5.3.3 Additional interoperability testing performed between KCC NSS Nokia Flexi BSS has been evaluated. (Annexure B - GSM-R IOT Declaration No. 2 by KCC and Nokia) Observation 1: The interfacing between Nokia BSS and KCC NSS is limited to "A" interface which is an open interface defined by 3GPP.” (d) Systra’s report p. 12 provides the following general conclusions: “6. CONCLUSION The analysis of the different certificates and documents presented for consultation to SYSTRA has enabled to assess the compatibility between KCC Core and Nokia Access network for the Indian GSM- R project. Based on the above information provided and based on the consultations done with other GSM-R experts, we have identified',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000,  0.5788, -0.0718],
#         [ 0.5788,  1.0000, -0.0618],
#         [-0.0718, -0.0618,  1.0000]])

<!-- ### Direct Usage (Transformers) <details><summary>Click to see the direct usage in Transformers</summary> </details> --> <!-- ### Downstream Usage (Sentence Transformers) You can finetune this model on your own dataset. <details><summary>Click to expand</summary> </details> --> <!-- ### Out-of-Scope Use *List how the model may foreseeably be misused and address what users ought not to do with the model.* --> <!-- ## Bias, Risks and Limitations *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* --> <!-- ### Recommendations *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* -->

Training Details

Training Datasets

train

• Dataset: train
• Size: 3,793 training samples
• Columns: <code>anchor</code> and <code>positive</code>
• Approximate statistics based on the first 1000 samples:

  	anchor	positive
  type	string	string
  details	<ul><li>min: 7 tokens</li><li>mean: 51.19 tokens</li><li>max: 146 tokens</li></ul>	<ul><li>min: 84 tokens</li><li>mean: 479.44 tokens</li><li>max: 757 tokens</li></ul>

• Samples:

  anchor	positive
  <code>What is the Sojitz-L&T Consortium's response to the Engineer's evaluation of 911 days of concurrent delay for MS-3, exceeding the notified EOT of 608 days, and where in the provided documentation can this response be found?</code>	<code>wawhhes he arkA Govt. of India (Ministry of Railways) EnterpriseNo. JP/EN/SLT/DAB/36Dated: 0§ 05,2020To,Authorised Representative Sojitz-L&T Consortium, Jaipur, 302021.Sub: - Design and Construction of Combined Package CTP-1& CTP-2 for Rewari - Iqbalgarh section of WDFC (CA No. HQ/EN/PWC/Phase-I/PK G-1&2/D&B/1/Sojitz-L&T) — Dispute No. 03 — Meeting for Amicable Settlement of Dispute under Clause 20.5 of GCC pursuant to DAB award dated 31.12.2019 held on 12.02.2020.Ref:- 1. This office letter No. JP/EN/SLT/DAB-3/36 dt. 03-01 -20202. C.0 letter no. 2017/HQ/EN/MA/WC/DAB/CTP1 &2/D-1,2,3/Pt. 1 dt 05-02-2020 3. C.O. letter no. 2017/HQ/EN/MA/WC/DAB/CTP1&2/D-1,2,3/Pt.1 dt 24-02-2020 4. Engineer’s letter no dated L-NKC-DFCC-PMC-2003-97 dayed 16.03.2020.With reference to DAB decision dated 31-12-2019 on Dispute-3, a meeting regarding Amicable Settlement of Dispute under Clause 20.5 of GCC. As per para 2 of said MoM, the Engineer is required to provide the impact analysis of Contractor’s concurre...</code>
  <code>What are the key project management consultancy services provided for the construction of the double line electrified railway track of Phase 1 of the Western Dedicated Freight Corridor, as indicated in the excerpt?</code>	<code>Registered Office :- 5th Floor, Pragati Maidan, Metro Station Building Complex, New Delhi - 110001 Regional Office : - C-16, Khushi Vihar, Patrakar Colony, Mansarovar, Jaipur — 302020 (Rajasthan) CIN : U60232DL2006GOI1 55068, Tel. +91- 141-2973543, Fax : 91- 141-2973542, Web : www.dfccil.gov.in©) NK CONSORTIMwith Signalling& Telecommunication system and related infrastructure for Rewari- Vadodara section Project Management Consultancy Services for Construction of Double Line Electrified Railway Track of Phase 1 of the Western Dedicated Freight Corridor</code>
  <code>What is the total EOT (Extension of Time) assessed and notified to the contractor for delays accrued up to 31-08-2016, according to the Amicable Settlement under FIDIC Cl. 20.5 for Contract Agreement No. HQ/EN/PWC/PMC-1?</code>	<code>Address: 4" Floor, PragatiMaidan, Metro Station Building, New Delhi - 110001, INDIATel: 91-11-233797 11/12 Fax: 91-11-23379710Nippon Koei Co., Ltd. — Oriental Consultants Co., Ltd. — Japan Transportation Consultants inc. — Ni on Koei India Pvt. Ltd. - Oriental Consultants india Pvt. Ltd. — Rites Ltd.Our Ref. _L-NKC-DFCC-PMC-2003-97Mr. Anurag Sharma GM/Coordination, DFCCIL, JaipurSubject: Project Management Consultancy Services for construction of double Line electrified railway track with signaling & telecommunication system and related infrastructure for Rewari — Vadodara section of Phase’ of the Western Dedicated Freight Corridor - Contract Agreement No. HQ/EN/PWC/PMC-1 dated 27.03.2014 - Assessment of Concurrent Delays attributable to the Contractor for EOT assessed and notified for the delays accrued upto 31-08-2016 as part of Amicable Settlement under FIDIC Cl. 20.5.</code>

• Loss: <code>MultipleNegativesRankingLoss</code> with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

test

• Dataset: test
• Size: 835 training samples
• Columns: <code>anchor</code> and <code>positive</code>
• Approximate statistics based on the first 835 samples:

  	anchor	positive
  type	string	string
  details	<ul><li>min: 21 tokens</li><li>mean: 48.39 tokens</li><li>max: 99 tokens</li></ul>	<ul><li>min: 85 tokens</li><li>mean: 318.32 tokens</li><li>max: 758 tokens</li></ul>

• Samples:

  anchor	positive
  <code>Given the outlined procedures for site procurement (8.2) and potential damages for delays (8.4), what recourse does the contractor have if the 'Site to be free from Encumbrances' clause (8.5) is breached, leading to project delays and increased costs?</code>	<code>Release of Performance Security7.5 Retention Moneymorth/epc/nh/37010/4/2010136368 Right of Way 8.1 The Site 8.2 Procurement of the Site 8.3 Damages for delay in handing over the Site 8.4 Site to be free from Encumbrances 8.5 Protection of Site from encroachments 8.6 Special/temporary Right of Way 8.7 Access to the Authority and the Authority’s Engineer 8.8 Geological and archaeological finds 9 Utilities and Trees 9.1 Existing utilities and roads 9.2 Shifting of obstructing utilities 9.3 New utilities 9.4 Felling of trees 10 Design and Construction of the Project Highway 10.1 Obligations prior to commencement of Works 10.2 Design and Drawings 10.3 Construction of the Project Highway 10.4 Maintenance during Construction Period 10.5 Extension of time for completion 10.6 Incomplete Works 10.7 Maintenance Manual 11 Quality Assurance, Monitoring and Supervision 11.1 Quality of Materials and workmanship 11.2 Quality control system 11.3 Methodology 11.4 Inspection and technical audit by the Au...</code>
  <code>In the context of assessing potential liquidated damages, what specific 'Maintenance Requirements' outlined in section 14.2 could be most challenging for the Contractor to consistently meet, and how would non-performance be objectively measured per section 14.6?</code>	<code>Completion Certificate12.1Tests on completion12.2 Provisional Certificate12.3 Completion of remaining Works12.4Completion Certificate12.5Rescheduling of Tests13Change of Scope13.1Change of Scope13.2Procedure for Change of Scope13.3Payment for Change of Scope13.4Restrictions on Change of Scope13.5Power of the Authority to undertake worksmorth/epc/nh/37010/4/201060606062626263636365656514 Maintenance 14.1 Maintenance obligations of the Contractor 14.2 Maintenance Requirements 14.3 Maintenance Programme 14.4 Safety, vehicle breakdowns and accidents 14.5 Lane closure 14.6 Reduction of payment for non-performance of Maintenance obligations 14.7 Authority’s right to take remedial measures 14.8 Restoration of loss or damage to Project Highway 14.9 Overriding powers of the Authority 15 Supervision and Monitoring during Maintenance 15.1 Inspection by the Contractor 15.2 Inspection and payments 15.3 Tests 15.4 Reports of unusual occurrence 16 Traffic Regulation 16.1 Traffic regulation by the Con...</code>
  <code>Regarding early completion, what specific criteria or metrics determine the bonus amount as per section 20?</code>	<code>Stage Payment Statement for Works19.5 Stage Payment for Works19.6 Monthly Maintenance Statement of the Project Highway19.7Payment for Maintenance of the Project Highway19.8 Payment of Damages19.9Time of payment and interest19.10. Price adjustment for the Works19.11 Restrictions on price adjustment19.12. Price adjustment for Maintenance of Project Highway19.13 Final Payment Statement19.14 Discharge19.15 Final Payment Certificatemorth/epc/nh/37010/4/201085858686878788919292939319.16 Final payment statement for Maintenance 19.17 Change in law 19.18 Correction of Interim Payment Certificates 19.19 Authority’s claims 19.20 Bonus for early completion 20 20.1 Insurance Insurance for Works and Maintenance 20.2 Notice to the Authority 20.3 Evidence of Insurance Cover 20.4 Remedy for failure to insure 20.5 Waiver of subrogation 20.6 Contractor’s waiver 20.7 Cross liabilities 20.8 Accident or injury to workmen 20.9 Insurance against accident to workmen 20.10 Application of insurance proceeds 20.1...</code>

• Loss: <code>MultipleNegativesRankingLoss</code> with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 12
• gradient_accumulation_steps: 5
• learning_rate: 2e-05
• num_train_epochs: 2
• lr_scheduler_type: cosine
• warmup_ratio: 0.1
• fp16: True
• optim: adamw_8bit
• gradient_checkpointing: True
• batch_sampler: no_duplicates

All Hyperparameters

<details><summary>Click to expand</summary>

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 12
• per_device_eval_batch_size: 8
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 5
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 2
• max_steps: -1
• lr_scheduler_type: cosine
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_8bit
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: True
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

</details>

Training Logs

Framework Versions

• Python: 3.12.12
• Sentence Transformers: 5.2.0
• Transformers: 4.56.2
• PyTorch: 2.9.0+cu126
• Accelerate: 1.12.0
• Datasets: 4.3.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

<!-- ## Glossary *Clearly define terms in order to be accessible across audiences.* --> <!-- ## Model Card Authors *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.* --> <!-- ## Model Card Contact *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.* -->