qwen3-bio-embedding-0.6b

SentenceTransformer

This is a sentence-transformers model trained on the json dataset. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer

<!-- - **Base model:** [Unknown](https://huggingface.co/unknown) -->

• Maximum Sequence Length: 32768 tokens
• Output Dimensionality: 1024 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • json

<!-- - **Language:** Unknown --> <!-- - **License:** Unknown -->

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 32768, 'do_lower_case': False, 'architecture': 'Qwen3Model'})
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': True, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'Instruct: Given a biomedical hypothesis, retrieve relevant scientific papers\nQuery: Chlamydia pneumoniae DNA in peripheral blood mononuclear cells is associated with cardiovascular disease.',
    'Chlamydia pneumoniae and atherosclerosis: critical assessment of diagnostic methods and relevance to treatment studies.\n\nA number of studies have found that inflammation of the vessel wall plays an essential role in both the initiation and progression of atherosclerosis and erosion and fissure and the eventual rupture of plaques. Chlamydia pneumoniae is one of the infectious agents that have been investigated as possible causes of this inflammation. Initial studies of the association of C. pneumoniae and cardiovascular disease (CVD) were seroepidemiologic, and these were followed by studies in which the organism was identified in vascular tissue from patients with CVD by electron microscopy, PCR and immunocytochemical staining (ICC). C. pneumoniae has also been isolated by culture from vascular tissue in a small number patients. However, no single serologic, PCR, or ICC assay has been used consistently across all studies. The assays used are also not standardized. Recent studies of serologic and PCR assays for diagnosis of C. pneumoniae infection have suggested that there may be substantial interlaboratory variation in the performance of these tests. It now appears that some of the inconsistency of results from study to study may be due, in part, to lack of standardized methods. Although initial seroepidemiologic studies demonstrated a significantly increased risk of adverse cardiac outcome in patients who were seropositive, subsequent prospective studies found either small or no increased risk. In addition to the lack of consistent serologic criteria, recent evaluations have demonstrated inherent problems with performance of the most widely used serologic methods. Most importantly, we do not have a reliable serologic marker for chronic or persistent C. pneumoniae infection.',
    '[Influence of depressive symptoms and gender in chronic low back pain rehabilitation outcome: a pilot study].\n\nCurrently, little is known about the influence of depressive symptoms and gender-specific aspects in rehabilitation outcome of patients with chronic low back pain. Effects of gender and depressive symptoms on rehabilitation outcome were examined immediately after rehabilitation, as well as three and six months after rehabilitation in 116 patients with chronic low back pain (43 women, 73 men; M=48 yrs.; ICD-10 diagnoses: M45.4/M45.5, M54.4/M54.5). Immediately after rehabilitation, general improvements with medium effect sizes in all rehabilitation measures were found. In contrast, six months after rehabilitation, only pain-related measures showed moderate improvements. Additionally, the mid-term outcomes were influenced by gender and depressive symptoms; women showed more stable rehabilitation outcomes in pain intensity, in the impaired function related to family/leisure, and the coping with pain strategies of "perceived self-competence" and "relaxation". In contrast, especially male patients with severe depressive symptoms revealed regressive rehabilitation outcomes, both in pain-related variables as well as marginally in the coping with pain strategy of "cognitive restructuring". In post-hoc analyses, in the mid-term, they even showed a deterioration of functional capacity and somatisation compared to prior to rehabilitation. Our results suggest that the outcome of orthopaedic rehabilitation may be persistently improved by implementing gender-specific treatments in general and elements of depression treatments for the patients with severe but sub-clinical depressive symptoms.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.8319, 0.0211],
#         [0.8319, 1.0000, 0.0233],
#         [0.0211, 0.0233, 1.0000]])

<!-- ### Direct Usage (Transformers) <details><summary>Click to see the direct usage in Transformers</summary> </details> --> <!-- ### Downstream Usage (Sentence Transformers) You can finetune this model on your own dataset. <details><summary>Click to expand</summary> </details> --> <!-- ### Out-of-Scope Use *List how the model may foreseeably be misused and address what users ought not to do with the model.* --> <!-- ## Bias, Risks and Limitations *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* --> <!-- ### Recommendations *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* -->

Training Details

Training Dataset

json

• Dataset: json
• Size: 4,858,688 training samples
• Columns: <code>anchor</code>, <code>positive</code>, and <code>negative</code>
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	<ul><li>min: 28 tokens</li><li>mean: 34.9 tokens</li><li>max: 56 tokens</li></ul>	<ul><li>min: 67 tokens</li><li>mean: 389.36 tokens</li><li>max: 980 tokens</li></ul>	<ul><li>min: 18 tokens</li><li>mean: 372.64 tokens</li><li>max: 1086 tokens</li></ul>

• Samples:

  anchor	positive	negative
  <code>Instruct: Given a biomedical hypothesis, retrieve relevant scientific papers<br>Query: Hookworm infection intensity is inversely related to haemoglobin levels in pregnant women.</code>	<code>Schistosoma mansoni in pregnancy and associations with anaemia in northwest Tanzania.<br><br>Schistosomiasis among pregnant women has been inadequately investigated. In order to determine the importance of Schistosoma mansoni in this subgroup, we conducted a cross-sectional survey of 972 women in Tanzania and investigated the prevalence of Schistosoma mansoni, hookworm and malaria and their associations with anaemia. Overall, 63.5% of women were infected with S. mansoni, with prevalence highest among younger women and decreasing with increasing age. The prevalence of hookworm was 56.3%, and 16.4% of women had malaria parasitaemia. Overall, 66.4% of women were anaemic. Increased risk of anaemia was associated with heavy infection with S. mansoni but not hookworm or Plasmodium falciparum parasitaemia.</code>	<code>Acute kidney injury in critically ill infants: the role of urine Neutrophil Gelatinase-Associated Lipocalin (NGAL).<br><br>Acute kidney injury (AKI) has emerged as an important health problem in the intensive care units, especially among infants delivered prematurely. Recent efforts to define and characterize AKI have led to studies of early AKI detection and will ultimately contribute to improvements in AKI outcomes. The discovery of biomarkers for AKI that might enable early recognition and clinical intervention to limit renal injury is therefore of intense contemporary interest. Neutrophil gelatinase-associated lipocalin (NGAL) is the most promising among all emerging markers for AKI; specifically, urine NGAL (uNGAL) predicts renal failure much earlier than serum creatinine. The recent availability of an automated immunoassay for measuring uNGAL in the clinical practice permits to introduce the test in emergency, having a turn around time (TAT) closely comparable with that of serum creati...</code>
  <code>Instruct: Given a biomedical hypothesis, retrieve relevant scientific papers<br>Query: Hookworm infection intensity is inversely related to haemoglobin levels in pregnant women.</code>	<code>Anemia and malaria at different altitudes in the western highlands of Kenya.<br><br>Malaria associated severe anemia in children is the most important complication of Plasmodium falciparum infection in sub-Saharan Africa. To evaluate anemia and malaria in an area with recurrent malaria epidemics in the western highlands of Kenya, we conducted cross-sectional surveys in four "lowland" (1440-1660 m) and two "highland" (1960 and 2040 m) villages in 2002. Among 1314 subjects randomly selected from all age groups, the overall prevalence of anemia (hemoglobin, Hb < 11 g/dl) was 14% and P. falciparum infection 17%. In children < or =5 years, anemia prevalence ranged from 57% at 1440 m to 11% at 2040 m and correlated with altitude (r = -0.88, P < 0.05). Similarly, P. falciparum prevalence ranged from 31 to 0% and correlated with altitude (r = -0.93, P < 0.01). Malnutrition defined by a body mass index <15th percentile characterized 39% of the population and the hookworm prevalence was 3.9%. In the l...</code>	<code>A high rate of durable responses with romidepsin, bortezomib, and dexamethasone in relapsed or refractory multiple myeloma.<br><br>We report results from a study exploring the combination of romidepsin, bortezomib, and dexamethasone for the treatment of patients with multiple myeloma (MM) previously treated with > 1 prior therapy. The primary objective was to determine the maximum tolerated dose (MTD) of the combination using a novel accelerated dose-escalation schedule in patients with relapsed or refractory MM. The secondary objective was to determine overall response (OR), time to progression (TTP), and overall survival (OS). The MTD identified was bortezomib 1.3 mg/m(2) (days 1, 4, 8, and 11), dexamethasone 20 mg (days 1, 2, 4, 5, 8, 9, 11, and 12), and romidepsin 10 mg/m(2) (days 1, 8, and 15) every 28 days. Thrombocytopenia (64%) was the most common ≥ grade 3 hematologic toxicity. Peripheral neuropathy occurred in 76% of patients (n = 19) (≥ grade 3, 8%; 95% confidence interval [CI] 1%...</code>
  <code>Instruct: Given a biomedical hypothesis, retrieve relevant scientific papers<br>Query: Hookworm infection intensity is inversely related to haemoglobin levels in pregnant women.</code>	<code>Hookworm (Necator americanus) infection and storage iron depletion.<br><br>The relationship between iron status and the intensity of infection with hookworm was investigated in a rural population on Karkar Island, Mandang Province, Papua New Guinea. There was a significant negative correlation between plasma ferritin level and hookworm burden, which was strongest in males. In contrast, there was no correlation between plasma ferritin and hookworm egg count, and no consistent correlation between haemoglobin level or haematocrit and either measure of hookworm intensity. The results suggest that the role of hookworm in the aetiology of anaemia may be difficult to assess without the accurate measurement of hookworm burden.</code>	<code>An easy-to-use online calculator to identify patients least likely to benefit from surgical resection of intrahepatic cholangiocarcinoma.<br><br>BACKGROUND AND OBJECTIVE: Although it is the primary curative-intent treatment option for intrahepatic cholangiocarcinoma, liver resection can be associated with high postoperative morbidity and mortality. As such, a high-risk resection may not be warranted when the oncological benefits are minimal. In the current study, we sought to develop 2 preoperative models to predict 90-day mortality and overall survival after liver resection for intrahepatic cholangiocarcinoma. METHODS: Patients who underwent curative-intent liver resection for intrahepatic cholangiocarcinoma between 1990 and 2020 were identified from an international multi-institutional database. Two prognostic models were developed with preoperative factors using multivariable regression analysis for 90-day mortality and overall survival. Patients were categorized into 3 risk groups: favor...</code>

• Loss: <code>MultipleNegativesRankingLoss</code> with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false,
      "directions": [
          "query_to_doc"
      ],
      "partition_mode": "joint",
      "hardness_mode": null,
      "hardness_strength": 0.0
  }
  

Evaluation Dataset

json

• Dataset: json
• Size: 1,207,774 evaluation samples
• Columns: <code>anchor</code>, <code>positive</code>, and <code>negative</code>
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	<ul><li>min: 26 tokens</li><li>mean: 36.28 tokens</li><li>max: 49 tokens</li></ul>	<ul><li>min: 45 tokens</li><li>mean: 400.11 tokens</li><li>max: 1719 tokens</li></ul>	<ul><li>min: 23 tokens</li><li>mean: 369.0 tokens</li><li>max: 1263 tokens</li></ul>

• Samples:

  anchor	positive	negative
  <code>Instruct: Given a biomedical hypothesis, retrieve relevant scientific papers<br>Query: Chlamydia pneumoniae DNA in peripheral blood mononuclear cells is associated with cardiovascular disease.</code>	<code>Chlamydia pneumoniae infection and mortality from ischaemic heart disease: large prospective study.<br><br>OBJECTIVE: To determine whether there is an independent association between infection with Chlamydia pneumoniae and ischaemic heart disease. DESIGN: Prospective study using a nested case-control design. SETTING: Medical centre in London run by BUPA, a private medical organisation. PARTICIPANTS: 21 520 professional men aged 35-64 who attended for a medical examination in London between 1975 and 1982. MAIN OUTCOME MEASURE: Death from ischaemic heart disease. RESULTS: The distributions of concentrations of IgG and IgA antibodies to C pneumoniae were similar in the 647 men who subsequently died of ischaemic heart disease and in 1294 age matched controls who did not. There was no material association with heart disease irrespective of the cut-off point chosen to define seropositivity. At a cut-off point that defines 15% of controls as positive, for example, the odds ratios were 1.26 (95% con...</code>	<code>Cerivastatin, a hydroxymethylglutaryl coenzyme A reductase inhibitor, inhibits cardiac myocyte hypertrophy induced by endothelin.<br><br>We investigated the direct effects of cerivastatin on hypertrophy of cultured rat neonatal myocytes induced by endothelin and the mechanism by which cerivastatin exerts its effects. Endothelin significantly increased [14C]phenylalanine ([14C]Phe) incorporation, atrial natriuretic peptide (ANP) release, ANP mRNA expression and cell size. Cerivastatin significantly reduced the increase in [14C]phenylalanine incorporation, ANP peptide release, ANP mRNA expression and cell size induced by endothelin, but pravastatin did not. Exogenous mevalonate completely prevented the inhibitory effect of cerivastatin on [14C]phenylalanine incorporation, ANP release and cell size. Cotreatment with geranylgeranyl pyrophosphate also attenuated the effect of cerivastatin on [14C]phenylalanine incorporation, but cotreatment with farnesyl pyrophosphate or squalene did not. Further...</code>
  <code>Instruct: Given a biomedical hypothesis, retrieve relevant scientific papers<br>Query: Chlamydia pneumoniae DNA in peripheral blood mononuclear cells is associated with cardiovascular disease.</code>	<code>Lack of association between prior infection with Chlamydia pneumoniae and acute or chronic coronary artery disease.<br><br>BACKGROUND: Higher than normal serologic titers and the detection of bacteria within atheroma have suggested an association between Chlamydia pneumoniae (C. pneumoniae) infection and coronary heart disease (CHD), but the relationship has not been well established. HYPOTHESIS: The study was designed to establish a lack of relationship between chronic C. pneumoniae infection and CHD. METHODS: Chlamydia-specific IgG-antibody was assayed using an indirect immunofluorescence test in the serum of 159 patients with severe arterial disease and 203 patients with a heart valve prostheses and no demonstrable CHD. Fatal and nonfatal vascular events and systemic thromboembolism were recorded over a 2-year period. RESULTS: In the arterial group 107 patients (67.3%) and in the valvular group 120/203 (59.1%) were positive for C. pneumoniae antibody. The number of patients with fatal or ...</code>	<code>Correlation Between Aspects of Perceived Patient Loneliness and Spinal Cord Stimulation Outcomes.<br><br>OBJECTIVES: Loneliness as a whole has been characterized as a health-related risk factor and is associated with worse outcomes after cardiac procedures. Evidence suggests that chronic pain patients are particularly vulnerable to feeling lonely. We examined the relationship between different aspects of loneliness and one-year postoperative outcomes after spinal cord stimulation (SCS) for chronic pain. MATERIALS AND METHODS: We contacted 69 patients with thoracic SCS who had participated in our prospective outcomes database with one-year follow-up to complete the validated, abbreviated UCLA Loneliness Scale (UCLA-3). We examined responses on question 9 of the Oswestry Disability Index (ODI), question 12 of the Beck Depression Inventory (BDI), and UCLA-3 due to their relevance to different aspects of loneliness. We conducted regression analyses to determine the relationship between aspects o...</code>
  <code>Instruct: Given a biomedical hypothesis, retrieve relevant scientific papers<br>Query: Chlamydia pneumoniae DNA in peripheral blood mononuclear cells is associated with cardiovascular disease.</code>	<code>Chlamydia pneumoniae infection and early asymptomatic carotid atherosclerosis.<br><br>BACKGROUND: Chronic Chlamydia pneumoniae infection has been implicated in the pathogenesis of atherosclerosis but whether it plays a role at an early stage in the disease is uncertain. An early estimate of atherosclerosis can be obtained by ultrasonic imaging of the carotid artery to determine intima-media thickness (IMT) and the thickness of any atheroma plaques. METHODS AND RESULTS: In 983 normal population individuals aged 30 to 70 years, we measured common carotid artery (CCA) and carotid bulb IMT, and also carotid plaque thickness and the degree of internal carotid artery (ICA) stenosis. C. pneumoniae IgA titers of >/=16 and IgG titers of >/=64 were taken as positive. There was no association between C. pneumoniae IgA or IgG seropositivity with right, left, or mean CCA or bulb IMT, or with the presence of carotid plaques. There was a significant association between IgA seropositivity and >50% mean caro...</code>	<code>The association between HIV infection and cervical cancer presentation and survival in Uganda.<br><br>Our objective was to determine how HIV infection impacts cervical cancer stage at presentation and overall survival (OS) among Ugandan women. This was a prospective study of 149 women diagnosed with cervical cancer from 2013 to 2015 at the Uganda Cancer Institute. Poisson regression models were fit to calculate prevalence ratios (PR) for the association between HIV infection and late stage at cancer diagnosis. The association between HIV infection and OS after cervical cancer diagnosis was evaluated using Cox proportional hazards models. The cohort included 53 HIV-positive and 96 HIV-negative participants. Median age at diagnosis was 44 years for HIV-positive and 54 years for HIV-negative participants. Seventy-seven percent of HIV-positive participants received antiretroviral therapy. Median baseline CD4 count was 373 cells/mm3 for HIV-positive participants versus 926 cells/mm3 for HIV-negat...</code>

• Loss: <code>MultipleNegativesRankingLoss</code> with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false,
      "directions": [
          "query_to_doc"
      ],
      "partition_mode": "joint",
      "hardness_mode": null,
      "hardness_strength": 0.0
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 128
• num_train_epochs: 5
• learning_rate: 2e-05
• warmup_steps: 0.1
• gradient_accumulation_steps: 2
• bf16: True
• eval_strategy: epoch
• per_device_eval_batch_size: 128
• dataloader_num_workers: 8
• ddp_find_unused_parameters: False

All Hyperparameters

<details><summary>Click to expand</summary>

• per_device_train_batch_size: 128
• num_train_epochs: 5
• max_steps: -1
• learning_rate: 2e-05
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_steps: 0.1
• optim: adamw_torch_fused
• optim_args: None
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• optim_target_modules: None
• gradient_accumulation_steps: 2
• average_tokens_across_devices: True
• max_grad_norm: 1.0
• label_smoothing_factor: 0.0
• bf16: True
• fp16: False
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• use_liger_kernel: False
• liger_kernel_config: None
• use_cache: False
• neftune_noise_alpha: None
• torch_empty_cache_steps: None
• auto_find_batch_size: False
• log_on_each_node: True
• logging_nan_inf_filter: True
• include_num_input_tokens_seen: no
• log_level: passive
• log_level_replica: warning
• disable_tqdm: False
• project: huggingface
• trackio_space_id: trackio
• eval_strategy: epoch
• per_device_eval_batch_size: 128
• prediction_loss_only: True
• eval_on_start: False
• eval_do_concat_batches: True
• eval_use_gather_object: False
• eval_accumulation_steps: None
• include_for_metrics: []
• batch_eval_metrics: False
• save_only_model: False
• save_on_each_node: False
• enable_jit_checkpoint: False
• push_to_hub: False
• hub_private_repo: None
• hub_model_id: None
• hub_strategy: every_save
• hub_always_push: False
• hub_revision: None
• load_best_model_at_end: False
• ignore_data_skip: False
• restore_callback_states_from_checkpoint: False
• full_determinism: False
• seed: 42
• data_seed: None
• use_cpu: False
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• dataloader_drop_last: True
• dataloader_num_workers: 8
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• dataloader_prefetch_factor: None
• remove_unused_columns: True
• label_names: None
• train_sampling_strategy: random
• length_column_name: length
• ddp_find_unused_parameters: False
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• ddp_backend: None
• ddp_timeout: 1800
• fsdp: []
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• deepspeed: None
• debug: []
• skip_memory_metrics: True
• do_predict: False
• resume_from_checkpoint: None
• warmup_ratio: None
• local_rank: -1
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

</details>

Training Logs

<details><summary>Click to expand</summary> </details>

Framework Versions

• Python: 3.13.5
• Sentence Transformers: 5.3.0
• Transformers: 5.3.0
• PyTorch: 2.10.0+cu128
• Accelerate: 1.13.0
• Datasets: 4.7.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{oord2019representationlearningcontrastivepredictive,
      title={Representation Learning with Contrastive Predictive Coding},
      author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
      year={2019},
      eprint={1807.03748},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/1807.03748},
}

<!-- ## Glossary *Clearly define terms in order to be accessible across audiences.* --> <!-- ## Model Card Authors *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.* --> <!-- ## Model Card Contact *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.* -->