ReasonIR-8B-scientific-challenge-v1.1

SentenceTransformer based on reasonir/ReasonIR-8B

This is a sentence-transformers model finetuned from reasonir/ReasonIR-8B. It maps sentences & paragraphs to a 4096-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: reasonir/ReasonIR-8B <!-- at revision c3d0690370ff4a8c3d3882d8dfa85c43650034fa -->
• Maximum Sequence Length: 131072 tokens
• Output Dimensionality: 4096 dimensions
• Similarity Function: Cosine Similarity

<!-- - **Training Dataset:** Unknown --> <!-- - **Language:** Unknown --> <!-- - **License:** Unknown -->

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 131072, 'do_lower_case': False, 'architecture': 'ReasonIRModel'})
  (1): Pooling({'word_embedding_dimension': 4096, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': False})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("shahafvl/ReasonIR-8B-scientific-challenge-v1.1")
# Run inference
sentences = [
    'Further investigations regarding stochastic fair queuing and new AQM algorithms are seen as desirable. In any case, network infrastructure updates will take time, particularly if the interest of the involved stakeholders is not aligned (as is often the case for network operators when dealing with over-the-top real-time traffic). It is, therefore, imperative that RTCWEB congestion control provides adequate improvement in the absence of any of the aforementioned schemes.',
    'Ensuring efficient and equitable management of network resources in the face of evolving traffic patterns is critical. Beyond the nuances of individual congestion control algorithms, a larger challenge exists: designing end-to-end traffic management mechanisms that can adapt to heterogeneous network conditions, diverse application requirements, and varying degrees of stakeholder alignment. Finding scalable solutions that promote both fairness and performance under these broader real-world constraints remains an open problem.',
    'Testing and evaluating active queue management algorithms often requires costly and complex emulation environments to faithfully reproduce real-world network conditions. The lack of standardized testing frameworks can hinder the comparability and reproducibility of research in this area.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 4096]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000,  0.6407, -0.0421],
#         [ 0.6407,  1.0000, -0.0686],
#         [-0.0421, -0.0686,  1.0000]])

<!-- ### Direct Usage (Transformers) <details><summary>Click to see the direct usage in Transformers</summary> </details> --> <!-- ### Downstream Usage (Sentence Transformers) You can finetune this model on your own dataset. <details><summary>Click to expand</summary> </details> --> <!-- ### Out-of-Scope Use *List how the model may foreseeably be misused and address what users ought not to do with the model.* -->

Evaluation

Metrics

Triplet

• Dataset: hierarchical_eval
• Evaluated with <code>TripletEvaluator</code>

<!-- ## Bias, Risks and Limitations *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* --> <!-- ### Recommendations *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* -->

Training Details

Training Dataset

Unnamed Dataset

• Size: 106,250 training samples
• Columns: <code>anchor</code>, <code>positive</code>, and <code>negative</code>
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	<ul><li>min: 45 tokens</li><li>mean: 92.1 tokens</li><li>max: 185 tokens</li></ul>	<ul><li>min: 42 tokens</li><li>mean: 65.17 tokens</li><li>max: 104 tokens</li></ul>	<ul><li>min: 25 tokens</li><li>mean: 40.35 tokens</li><li>max: 61 tokens</li></ul>

• Samples:

  anchor	positive	negative
  <code>The only evidence that factoring is hard consists of our failure so far to find a fast and practical factoring algorithm. ( The polynomial-time factoring algorithms that are based on the use of quantum computers are not considered to be practical and not addressed in this survey.) Interestingly, and to an outsider maybe surprisingly, an entire industry is based on this belief that factoring is hard: the security, LENSTRA i.e., the unbreakability, of one of the most popular public key cryptosystems relies on the supposed difficulty of factoring (cf.</code>	<code>The perceived computational difficulty of certain mathematical problems forms the foundation for the security of many cryptographic protocols. This reliance exists despite the absence of formal proofs confirming that these problems are inherently hard to solve efficiently. The broader challenge is to understand whether basing security on presumed hardness is justified, and what the risks are if future advances in algorithms or computing architectures undermine these assumptions.</code>	<code>Developing methods to accurately predict when a factoring algorithm will outperform existing approaches is a significant challenge in the optimization of cryptographic systems. Such prediction mechanisms would help practitioners remain ahead of unforeseen breakthroughs in algorithmic factoring techniques.</code>
  <code>The only evidence that factoring is hard consists of our failure so far to find a fast and practical factoring algorithm. ( The polynomial-time factoring algorithms that are based on the use of quantum computers are not considered to be practical and not addressed in this survey.) Interestingly, and to an outsider maybe surprisingly, an entire industry is based on this belief that factoring is hard: the security, LENSTRA i.e., the unbreakability, of one of the most popular public key cryptosystems relies on the supposed difficulty of factoring (cf.</code>	<code>The perceived computational difficulty of certain mathematical problems forms the foundation for the security of many cryptographic protocols. This reliance exists despite the absence of formal proofs confirming that these problems are inherently hard to solve efficiently. The broader challenge is to understand whether basing security on presumed hardness is justified, and what the risks are if future advances in algorithms or computing architectures undermine these assumptions.</code>	<code>A key challenge in algorithm design relates to selecting the most efficient approach for a given hardware architecture. The relative performance of factoring algorithms can vary dramatically between CPUs, GPUs, and specialized hardware, making portability and benchmarking difficult.</code>
  <code>The only evidence that factoring is hard consists of our failure so far to find a fast and practical factoring algorithm. ( The polynomial-time factoring algorithms that are based on the use of quantum computers are not considered to be practical and not addressed in this survey.) Interestingly, and to an outsider maybe surprisingly, an entire industry is based on this belief that factoring is hard: the security, LENSTRA i.e., the unbreakability, of one of the most popular public key cryptosystems relies on the supposed difficulty of factoring (cf.</code>	<code>The perceived computational difficulty of certain mathematical problems forms the foundation for the security of many cryptographic protocols. This reliance exists despite the absence of formal proofs confirming that these problems are inherently hard to solve efficiently. The broader challenge is to understand whether basing security on presumed hardness is justified, and what the risks are if future advances in algorithms or computing architectures undermine these assumptions.</code>	<code>Ensuring the robustness of cryptographic systems to implementation flaws, such as side-channel attacks, remains an ongoing challenge distinct from the underlying mathematical difficulty of factoring. Adversaries may exploit vulnerabilities in physical or software implementations regardless of the computational hardness of the underlying problem.</code>

• Loss: <code>MultipleNegativesRankingLoss</code> with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

Evaluation Dataset

Unnamed Dataset

• Size: 18,750 evaluation samples
• Columns: <code>anchor</code>, <code>positive</code>, and <code>negative</code>
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	<ul><li>min: 44 tokens</li><li>mean: 88.85 tokens</li><li>max: 173 tokens</li></ul>	<ul><li>min: 45 tokens</li><li>mean: 64.61 tokens</li><li>max: 99 tokens</li></ul>	<ul><li>min: 24 tokens</li><li>mean: 40.94 tokens</li><li>max: 73 tokens</li></ul>

• Samples:

  anchor	positive	negative
  <code>On the other hand, original data, such as biomedical data, cannot be easily collected due to privacy and security concerns. Therefore, researchers interested in solving the original task are unable to collect a sufficient amount of data to train DNNs. Conventional methods address this problem by applying transfer learning.</code>	<code>In many fields, the procurement of large-scale, high-quality datasets is hindered by issues such as privacy, proprietary restrictions, and logistical constraints. As a result, the effectiveness of deep learning models is restricted by limited access to relevant training data. Broadly, the challenge lies in developing machine learning methods that can perform well despite data scarcity caused by various collection barriers, beyond just privacy and security concerns.</code>	<code>Standard transfer learning approaches can result in negative transfer if the source and target domains are insufficiently related, leading to degraded performance rather than improvements in the original task.</code>
  <code>On the other hand, original data, such as biomedical data, cannot be easily collected due to privacy and security concerns. Therefore, researchers interested in solving the original task are unable to collect a sufficient amount of data to train DNNs. Conventional methods address this problem by applying transfer learning.</code>	<code>In many fields, the procurement of large-scale, high-quality datasets is hindered by issues such as privacy, proprietary restrictions, and logistical constraints. As a result, the effectiveness of deep learning models is restricted by limited access to relevant training data. Broadly, the challenge lies in developing machine learning methods that can perform well despite data scarcity caused by various collection barriers, beyond just privacy and security concerns.</code>	<code>When dealing with biomedical data, ensuring that trained models do not inadvertently memorize sensitive information is a major concern, requiring the development of privacy-preserving techniques such as differential privacy or federated learning.</code>
  <code>On the other hand, original data, such as biomedical data, cannot be easily collected due to privacy and security concerns. Therefore, researchers interested in solving the original task are unable to collect a sufficient amount of data to train DNNs. Conventional methods address this problem by applying transfer learning.</code>	<code>In many fields, the procurement of large-scale, high-quality datasets is hindered by issues such as privacy, proprietary restrictions, and logistical constraints. As a result, the effectiveness of deep learning models is restricted by limited access to relevant training data. Broadly, the challenge lies in developing machine learning methods that can perform well despite data scarcity caused by various collection barriers, beyond just privacy and security concerns.</code>	<code>Another challenge arises when the available biomedical datasets are heavily imbalanced, with certain classes being overrepresented and others underrepresented, complicating the training and evaluation of reliable deep neural networks.</code>

• Loss: <code>MultipleNegativesRankingLoss</code> with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• learning_rate: 2e-05
• num_train_epochs: 1
• warmup_ratio: 0.1
• bf16: True
• load_best_model_at_end: True
• push_to_hub: True
• hub_model_id: shahafvl/ReasonIR-8B-scientific-challenge-v1.1
• hub_private_repo: True
• auto_find_batch_size: True

All Hyperparameters

<details><summary>Click to expand</summary>

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 8
• per_device_eval_batch_size: 8
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: True
• resume_from_checkpoint: None
• hub_model_id: shahafvl/ReasonIR-8B-scientific-challenge-v1.1
• hub_strategy: every_save
• hub_private_repo: True
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: True
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

</details>

Training Logs

Framework Versions

• Python: 3.11.9
• Sentence Transformers: 5.0.0
• Transformers: 4.54.0
• PyTorch: 2.7.1+cu126
• Accelerate: 1.9.0
• Datasets: 4.0.0
• Tokenizers: 0.21.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

<!-- ## Glossary *Clearly define terms in order to be accessible across audiences.* --> <!-- ## Model Card Authors *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.* --> <!-- ## Model Card Contact *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.* -->