finetuned_rag_retriever

SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2

This is a sentence-transformers model finetuned from sentence-transformers/all-MiniLM-L6-v2. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/all-MiniLM-L6-v2 <!-- at revision c9745ed1d9f207416be6d2e6f8de32d1f16199bf -->
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity

<!-- - **Training Dataset:** Unknown --> <!-- - **Language:** Unknown --> <!-- - **License:** Unknown -->

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("OmarIDK/finetuned_rag_retriever")
# Run inference
sentences = [
    'What are the steps involved in designing small molecule drugs targeting respiratory disease pathways such as bronchoconstriction and airway inflammation?',
    "How can we design small molecule drugs targeting specific respiratory disease pathways, such as bronchoconstriction or airway inflammation, using medicinal chemistry approaches?\n\nDesigning small molecule drugs targeting specific respiratory disease pathways, such as bronchoconstriction or airway inflammation, can be achieved through a systematic medicinal chemistry approach. This involves several key steps:\n\n1. Target identification and validation: The first step is to identify and validate the molecular targets involved in the respiratory disease pathways. These targets can be proteins, enzymes, or receptors that play a crucial role in bronchoconstriction or airway inflammation. Examples of such targets include beta-2 adrenergic receptors, muscarinic receptors, and leukotriene receptors.\n\n2. Hit identification: Once the target is identified, the next step is to find small molecules that can interact with the target and modulate its activity. This can be done through various techniques, such as high-throughput screening (HTS) of compound libraries, fragment-based drug discovery, or in silico (computer-aided) drug design.\n\n3. Hit-to-lead optimization: After identifying the initial hits, medicinal chemists optimize these compounds to improve their potency, selectivity, and drug-like properties. This involves synthesizing and testing a series of analogs with slight modifications in their chemical structure to identify the structure-activity relationship (SAR). The goal is to find a lead compound with the desired biological activity and minimal off-target effects.\n\n4. Lead optimization: The lead compound is further optimized to enhance its pharmacokinetic (PK) and pharmacodynamic (PD) properties, such as absorption, distribution, metabolism, excretion, and toxicity (ADMET). This step involves fine-tuning the chemical structure to improve the drug's overall performance, including its solubility, stability, and bioavailability.\n\n5. Preclinical testing: The optimized lead compound undergoes extensive preclinical testing in vitro (cell-based assays) and in vivo (animal models) to evaluate its safety, efficacy, and mechanism of action. This helps to determine the compound's suitability for further development as a drug candidate.\n\n6. Clinical trials: If the preclinical data are promising, the drug candidate proceeds to clinical trials, where its safety and efficacy are tested in human subjects. This involves a phased approach, starting with Phase I (safety and dosage), Phase II (efficacy and side effects), and Phase III (comparison with existing treatments) trials.\n\n7. Regulatory approval and post-marketing surveillance: If the drug candidate demonstrates safety and efficacy in clinical trials, it can be submitted for regulatory approval (e.g., by the FDA). Once approved, the drug is marketed and subjected to post-marketing surveillance to monitor its long-term safety and effectiveness in the general population.\n\nIn summary, designing small molecule drugs",
    'The solar system consists of the Sun and all celestial bodies that are bound to it by gravity, including eight planets, their moons, and various smaller objects such as dwarf planets and asteroids. The planets in order from the Sun are Mercury, Venus, Earth, Mars, Jupiter, Saturn, Uranus, and Neptune. Each planet has unique characteristics, including its composition, atmosphere, and surface conditions. For instance, Jupiter is known for its Great Red Spot, a giant storm, while Saturn is famous for its prominent ring system. The study of the solar system helps scientists understand planetary formation, the potential for life on other planets, and the history of our own planet Earth.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

<!-- ### Direct Usage (Transformers) <details><summary>Click to see the direct usage in Transformers</summary> </details> --> <!-- ### Downstream Usage (Sentence Transformers) You can finetune this model on your own dataset. <details><summary>Click to expand</summary> </details> --> <!-- ### Out-of-Scope Use *List how the model may foreseeably be misused and address what users ought not to do with the model.* --> <!-- ## Bias, Risks and Limitations *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* --> <!-- ### Recommendations *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* -->

Training Details

Training Dataset

Unnamed Dataset

• Size: 498 training samples
• Columns: <code>sentence_0</code>, <code>sentence_1</code>, and <code>sentence_2</code>
• Approximate statistics based on the first 498 samples:

  	sentence_0	sentence_1	sentence_2
  type	string	string	string
  details	<ul><li>min: 10 tokens</li><li>mean: 27.13 tokens</li><li>max: 50 tokens</li></ul>	<ul><li>min: 17 tokens</li><li>mean: 248.96 tokens</li><li>max: 256 tokens</li></ul>	<ul><li>min: 46 tokens</li><li>mean: 162.92 tokens</li><li>max: 256 tokens</li></ul>

• Samples:

  sentence_0	sentence_1	sentence_2
  <code>What is the heat of crystallization of sodium acetate trihydrate when dissolved in water and cooled?</code>	<code>What is the heat of crystallization of 50 g of sodium acetate trihydrate (NaC2H3O2.3H2O) when it is dissolved in 200 mL of water at 25°C and then the resulting solution is cooled to 10°C, assuming that no heat is lost to the surroundings? The heat of solution of sodium acetate trihydrate is -4245 J/g. <br><br>To find the heat of crystallization of sodium acetate trihydrate, we first need to calculate the heat released when it is dissolved in water and then the heat absorbed when the solution is cooled.<br><br>1. Heat released during dissolution:<br>Heat of solution = -4245 J/g<br>Mass of sodium acetate trihydrate = 50 g<br><br>Heat released = Heat of solution × Mass<br>Heat released = -4245 J/g × 50 g = -212250 J<br><br>Since the heat released is negative, it means that the dissolution process is exothermic and releases heat to the surroundings.<br><br>2. Heat absorbed during cooling:<br>We will use the formula q = mcΔT, where q is the heat absorbed, m is the mass of the solution, c is the specific heat capacity of water (4.1...</code>	<code>The process of cellular respiration is vital for the survival of most living organisms. It involves the conversion of glucose and oxygen into energy, carbon dioxide, and water. This metabolic process occurs in several stages, including glycolysis, the citric acid cycle, and oxidative phosphorylation. During glycolysis, glucose is broken down into pyruvate, yielding a small amount of ATP. The citric acid cycle then processes pyruvate to produce electron carriers such as NADH and FADH2, which are crucial for the final stage of respiration. In oxidative phosphorylation, these carriers donate electrons to the electron transport chain, ultimately leading to the production of a large amount of ATP. Understanding cellular respiration is essential for studies in biology and medicine, as it provides insights into energy production, metabolic diseases, and overall cell function.</code>
  <code>How can the concentration of copper (II) ions in a water sample be determined using colorimetry?</code>	<code>How can the concentration of copper (II) ions in a water sample be quantified using colorimetry?<br><br>Colorimetry is a technique used to determine the concentration of a colored compound in a solution by measuring the absorbance of light at a specific wavelength. In this case, we want to quantify the concentration of copper (II) ions in a water sample. To do this, we can follow these steps:<br><br>1. Prepare a series of copper (II) ion standards: Prepare a stock solution of known concentration of copper (II) sulfate (CuSO4). Then, create a series of standard solutions with varying concentrations of copper (II) ions by diluting the stock solution with distilled water. For example, you can prepare 5 standard solutions with concentrations of 1, 2, 4, 6, and 8 ppm (parts per million).<br><br>2. Complex formation: Copper (II) ions in solution are not very colorful and do not strongly absorb visible light. To enhance the color and improve the sensitivity of the measurement, we can form a complex with a suit...</code>	<code>The process of osmosis is crucial for maintaining the balance of fluids in biological systems. It involves the movement of water molecules through a semi-permeable membrane from an area of lower solute concentration to an area of higher solute concentration. This process is essential for cell function, as it helps regulate hydration, nutrient absorption, and waste removal. Osmosis plays a vital role in various physiological processes, including the operation of kidneys and the overall homeostasis of organisms. Understanding osmosis is important in fields such as biology, medicine, and environmental science.</code>
  <code>How can biotechnology be utilized to create drought-resistant crops and improve their yield and resilience in varying environmental conditions?</code>	<code>Biotechnology can be used to create drought-resistant crops through various approaches, including genetic engineering, marker-assisted breeding, and gene editing techniques. These methods can help increase the yield and overall resilience of crops in different environmental conditions. Some specific methods are:<br><br>1. Genetic engineering: This involves the direct manipulation of an organism's genes using biotechnology. Scientists can introduce specific genes from drought-resistant plants or other organisms into the target crop, conferring drought tolerance. For example, the introduction of the DREB (dehydration-responsive element-binding) gene family, which regulates the expression of drought-responsive genes, has been shown to improve drought tolerance in crops like rice and wheat.<br><br>2. Marker-assisted breeding: This technique combines traditional plant breeding with molecular biology. It involves identifying specific DNA markers associated with drought tolerance traits in plants and usi...</code>	<code>The process of cellular respiration is crucial for all aerobic organisms, as it allows them to convert biochemical energy from nutrients into adenosine triphosphate (ATP). This process occurs in several stages:<br><br>1. Glycolysis: This stage takes place in the cytoplasm and breaks down glucose into pyruvate, producing a small amount of ATP and NADH.<br><br>2. Krebs Cycle: Also known as the citric acid cycle, this stage occurs in the mitochondria, where pyruvate is further broken down, releasing carbon dioxide and transferring high-energy electrons to carriers like NADH and FADH₂.<br><br>3. Electron Transport Chain: The electrons from NADH and FADH₂ are passed through a series of proteins in the mitochondrial membrane, leading to the production of a large amount of ATP and the reduction of oxygen to form water.<br><br>4. Anaerobic Respiration: In the absence of oxygen, some organisms can undergo anaerobic respiration, which allows them to generate energy through processes like fermentation, but less efficien...</code>

• Loss: <code>TripletLoss</code> with these parameters:

  {
      "distance_metric": "TripletDistanceMetric.EUCLIDEAN",
      "triplet_margin": 5
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

<details><summary>Click to expand</summary>

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 3
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• tp_size: 0
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin

</details>

Framework Versions

• Python: 3.12.8
• Sentence Transformers: 3.4.1
• Transformers: 4.51.3
• PyTorch: 2.5.1+cu124
• Accelerate: 1.3.0
• Datasets: 3.2.0
• Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

TripletLoss

@misc{hermans2017defense,
    title={In Defense of the Triplet Loss for Person Re-Identification},
    author={Alexander Hermans and Lucas Beyer and Bastian Leibe},
    year={2017},
    eprint={1703.07737},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}

<!-- ## Glossary *Clearly define terms in order to be accessible across audiences.* --> <!-- ## Model Card Authors *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.* --> <!-- ## Model Card Contact *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.* -->