Downloads · 30 days
9
4% of all-time downloads
FareedKhan/flax-sentence-embeddings_all_datasets_v4_MiniLM-L6_FareedKhan_prime_synthetic_data_2k_4_16
flax-sentence-embeddings_all_datasets_v4_MiniLM-L6_FareedKhan_prime_synthetic_data_2k_4_16 is a sentence similarity model from FareedKhan. Use it when you need a score for how close two texts are. It is set up for sentence-transformers.
This is a sentence-transformers model finetuned from flax-sentence-embeddings/alldatasetsv4MiniLM-L6 on the json dataset. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for sema…
Downloads · 30 days
9
4% of all-time downloads
All-time downloads
249
Public
Parameters
22.7M
90.9 MB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors90.9 MB · 99%
From the Hugging Face model README
This is a sentence-transformers model finetuned from flax-sentence-embeddings/all_datasets_v4_MiniLM-L6 on the json dataset. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
SentenceTransformer(
(0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: BertModel
(1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
(2): Normalize()
)
First install the Sentence Transformers library:
pip install -U sentence-transformers
Then you can load this model and run inference.
from sentence_transformers import SentenceTransformer
# Download from the 🤗 Hub
model = SentenceTransformer("FareedKhan/flax-sentence-embeddings_all_datasets_v4_MiniLM-L6_FareedKhan_prime_synthetic_data_2k_4_16")
# Run inference
sentences = [
'\nEpstein-Barr virus-associated mesenchymal tumor is a disease designated as a type of leiomyosarcoma within the disease nomenclature system MONDO. This specific condition is uniquely characterized by its association with the Epstein-Barr virus and exhibits symptoms commonly related to an underlying malignancy, such as fatigue, fever, and muscle pain. Identified as a subgroup of leiomyosarcoma, it also encompasses related diseases including Epstein-Barr virus-related tumor, follicular dendritic cell sarcoma, and myopericytoma, all of which share the hallmark of being influenced by the Epstein-Barr virus. This classification emphasizes the role of viral infection in the development and manifestation of these tumor types, offering insights into potential pathways of disease progression and suggesting avenues for targeted therapeutic interventions.',
'What type of leiomyosarcoma commonly manifests with fatigue, fever, and muscle pain?',
"Could you provide me with a list of medications that display synergistic effects when combined with Lemborexant, are prescribed for the same indications, and possess an elimination half-life close to 37 hours? I am interested in exploring alternative treatments compatible with Lemborexant's therapeutic indications that may offer extended efficacy through the prolonged half-life of the secondary drug when co-administered.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]
# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
<!--
### Direct Usage (Transformers)
<details><summary>Click to see the direct usage in Transformers</summary>
</details>
-->
<!--
### Downstream Usage (Sentence Transformers)
You can finetune this model on your own dataset.
<details><summary>Click to expand</summary>
</details>
-->
<!--
### Out-of-Scope Use
*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->
dim_384| Metric | Value |
|---|---|
| cosine_accuracy@1 | 0.3663 |
| cosine_accuracy@3 | 0.4406 |
| cosine_accuracy@5 | 0.4653 |
| cosine_accuracy@10 | 0.5149 |
| cosine_precision@1 | 0.3663 |
| cosine_precision@3 | 0.1469 |
| cosine_precision@5 | 0.0931 |
| cosine_precision@10 | 0.0515 |
| cosine_recall@1 | 0.3663 |
| cosine_recall@3 | 0.4406 |
| cosine_recall@5 | 0.4653 |
| cosine_recall@10 | 0.5149 |
| cosine_ndcg@10 | 0.437 |
| cosine_mrr@10 | 0.4127 |
| cosine_map@100 | 0.4187 |
| positive | anchor | |
|---|---|---|
| type | string | string |
| details | <ul><li>min: 2 tokens</li><li>mean: 117.23 tokens</li><li>max: 128 tokens</li></ul> | <ul><li>min: 13 tokens</li><li>mean: 35.72 tokens</li><li>max: 128 tokens</li></ul> |
| positive | anchor |
|---|---|
| <code><br>The list you've provided appears to include a wide range of substances and compounds from various fields such as chemistry, medicine, and pharmaceuticals. Here's a brief categorization and description of a few categories:<br><br>### Chemical Compounds and Their Uses<br><br>1. Calcin: An anticoagulant used to prevent blood clots by inhibiting the formation of calcium deposits in the blood.<br>2. Colistimethate: An antibiotic used to treat serious bacterial infections that are not responsive to other antibiotics.<br><br>### Medication and Drug Discovery<br><br>3. Benznidazole: An antiparasitic drug used to treat infections like American trypanosomiasis (Chagas' disease).<br>4. Amediplase: A promising stem cell derivative molecule developed as a cancer drug. It appears to have the potential to selectively target and kill cancer cells.<br><br>### Medical Imaging Agents<br><br>5. Gadodiamide, Gadoteridol, Iothalamic acid, Ioversol, and Technetium Tc-99m exametazime: These are contrast agents used in medical imaging to enhance the visibility of organs or tissues in MRI, CT, and other imaging scans.<br><br>### Drug Delivery and Coagulation Management<br><br>6. Kebuzone: Although not widely recognized, specific uses or pharmaceutical names like this might refer to a less common medication or generic name.<br>7. Robenacoxib: A selective COX-2 inhibitor used to manage pain and inflammation.<br>8. Melagatran: An anticoagulant, likely used in managing blood clotting.<br><br>### Immune Modulation and Cancer Therapy<br><br>9. **Idaruciz</code> | <code>Could you recommend any medications that effectively treat bacterial arthritis and are compatible with Alprostadil? Ideally, the medication should have a short half-life, being metabolized within an hour or so, to accommodate my active lifestyle.</code> |
| <code><br>MYT1, also known by various aliases such as 'C20orf36', 'MTF1', 'MYTI', 'NZF2', 'PLPB1', 'ZC2H2C1', and 'ZC2HC4A', is a gene/protein that codes for a zinc finger-containing DNA-binding protein. This gene is part of a family of neural-specific proteins that play a crucial role in the developing nervous system by binding to the promoter regions of proteolipid proteins in the central nervous system. The protein encoded by MYT1 has been associated with certain diseases, including oculo-auriculo-vertebral spectrum and hemifacial microsomia. It is involved in various cellular components such as the nucleus, chromatin, cytosol, and nucleoplasm, and its biological processes include cell differentiation, regulation of transcription by RNA polymerase II, and nervous system development. MYT1 is expressed in a wide variety of tissues, such as the pituitary gland, pituitary gland</code> | <code>Which gene or protein is not expressed in the stomach fundus and nasal cavity epithelial tissue?</code> |
| <code><br><br>### Key Information on Long QT Syndrome<br><br>#### What is Long QT Syndrome?<br>- Definition: A genetic condition that affects the heart's electrical system, causing abnormal heart rhythms.<br>- Signs & Symptoms: Syncope (fainting), seizures, abnormal heart rhythms, and sudden cardiac death.<br><br>#### Forms of Long QT Syndrome<br>- Congenital (Romano-Ward Syndrome): Occurs when a single gene variant is inherited from one parent, often with normal hearing.<br>- Acquired: Resulting from medications, medical conditions, or electrolyte imbalances.<br><br>#### Risk Factors<br>- Hereditary: A family history increases risk.<br>- Certain Medications: Common antibiotics, antifungals, and some antidepressants can cause it.<br>- Medical Conditions: Diarrhea, vomiting, eating disorders, or acute kidney injury.<br><br>#### Complications<br>- Torsades de Pointes: Chaotic heart rhythm that can be fatal.<br>- Ventricular Fibrillation: Rapid, ineffective heartbeats; can lead to sudden death without immediate treatment.<br>- Sudden Death: Occurs in an otherwise healthy individual.<br><br>#### Prevention<br>- Medication Review: Regularly check medications for</code> | <code>Which cardiac arrhythmia contraindicates the use of medications prescribed for bladder infections?</code> |
{
"loss": "MultipleNegativesRankingLoss",
"matryoshka_dims": [
384
],
"matryoshka_weights": [
1
],
"n_dims_per_step": -1
}
eval_strategy: epochper_device_train_batch_size: 16learning_rate: 1e-05num_train_epochs: 4warmup_ratio: 0.1bf16: Truetf32: Falseload_best_model_at_end: Trueoverwrite_output_dir: Falsedo_predict: Falseeval_strategy: epochprediction_loss_only: Trueper_device_train_batch_size: 16per_device_eval_batch_size: 8per_gpu_train_batch_size: Noneper_gpu_eval_batch_size: Nonegradient_accumulation_steps: 1eval_accumulation_steps: Nonetorch_empty_cache_steps: Nonelearning_rate: 1e-05weight_decay: 0.0adam_beta1: 0.9adam_beta2: 0.999adam_epsilon: 1e-08max_grad_norm: 1.0num_train_epochs: 4max_steps: -1lr_scheduler_type: linearlr_scheduler_kwargs: {}warmup_ratio: 0.1warmup_steps: 0log_level: passivelog_level_replica: warninglog_on_each_node: Truelogging_nan_inf_filter: Truesave_safetensors: Truesave_on_each_node: Falsesave_only_model: Falserestore_callback_states_from_checkpoint: Falseno_cuda: Falseuse_cpu: Falseuse_mps_device: Falseseed: 42data_seed: Nonejit_mode_eval: Falseuse_ipex: Falsebf16: Truefp16: Falsefp16_opt_level: O1half_precision_backend: autobf16_full_eval: Falsefp16_full_eval: Falsetf32: Falselocal_rank: 0ddp_backend: Nonetpu_num_cores: Nonetpu_metrics_debug: Falsedebug: []dataloader_drop_last: Falsedataloader_num_workers: 0dataloader_prefetch_factor: Nonepast_index: -1disable_tqdm: Falseremove_unused_columns: Truelabel_names: Noneload_best_model_at_end: Trueignore_data_skip: Falsefsdp: []fsdp_min_num_params: 0fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}fsdp_transformer_layer_cls_to_wrap: Noneaccelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}deepspeed: Nonelabel_smoothing_factor: 0.0optim: adamw_torchoptim_args: Noneadafactor: Falsegroup_by_length: Falselength_column_name: lengthddp_find_unused_parameters: Noneddp_bucket_cap_mb: Noneddp_broadcast_buffers: Falsedataloader_pin_memory: Truedataloader_persistent_workers: Falseskip_memory_metrics: Trueuse_legacy_prediction_loop: Falsepush_to_hub: Falseresume_from_checkpoint: Nonehub_model_id: Nonehub_strategy: every_savehub_private_repo: Falsehub_always_push: Falsegradient_checkpointing: Falsegradient_checkpointing_kwargs: Noneinclude_inputs_for_metrics: Falseeval_do_concat_batches: Truefp16_backend: autopush_to_hub_model_id: Nonepush_to_hub_organization: Nonemp_parameters:auto_find_batch_size: Falsefull_determinism: Falsetorchdynamo: Noneray_scope: lastddp_timeout: 1800torch_compile: Falsetorch_compile_backend: Nonetorch_compile_mode: Nonedispatch_batches: Nonesplit_batches: Noneinclude_tokens_per_second: Falseinclude_num_input_tokens_seen: Falseneftune_noise_alpha: Noneoptim_target_modules: Nonebatch_eval_metrics: Falseeval_on_start: Falseuse_liger_kernel: Falseeval_use_gather_object: Falsebatch_sampler: batch_samplermulti_dataset_batch_sampler: proportional| Epoch | Step | Training Loss | dim_384_cosine_map@100 |
|---|---|---|---|
| 0 | 0 | - | 0.3971 |
| 0.0877 | 10 | 1.5497 | - |
| 0.1754 | 20 | 1.334 | - |
| 0.2632 | 30 | 1.2332 | - |
| 0.3509 | 40 | 1.1818 | - |
| 0.4386 | 50 | 1.087 | - |
| 0.5263 | 60 | 1.2103 | - |
| 0.6140 | 70 | 1.1323 | - |
| 0.7018 | 80 | 1.0869 | - |
| 0.7895 | 90 | 0.9275 | - |
| 0.8772 | 100 | 1.0684 | - |
| 0.9649 | 110 | 0.9702 | - |
| 1.0 | 114 | - | 0.4142 |
| 1.0526 | 120 | 1.0792 | - |
| 1.1404 | 130 | 1.1194 | - |
| 1.2281 | 140 | 0.9212 | - |
| 1.3158 | 150 | 1.0393 | - |
| 1.4035 | 160 | 1.099 | - |
| 1.4912 | 170 | 0.8902 | - |
| 1.5789 | 180 | 0.854 | - |
| 1.6667 | 190 | 0.6828 | - |
| 1.7544 | 200 | 0.9187 | - |
| 1.8421 | 210 | 0.8597 | - |
| 1.9298 | 220 | 1.0286 | - |
| 2.0 | 228 | - | 0.4179 |
| 2.0175 | 230 | 0.6874 | - |
| 2.1053 | 240 | 0.7523 | - |
| 2.1930 | 250 | 0.7594 | - |
| 2.2807 | 260 | 0.6929 | - |
| 2.3684 | 270 | 0.7718 | - |
| 2.4561 | 280 | 0.7803 | - |
| 2.5439 | 290 | 0.7324 | - |
| 2.6316 | 300 | 0.7252 | - |
| 2.7193 | 310 | 0.7532 | - |
| 2.8070 | 320 | 0.8368 | - |
| 2.8947 | 330 | 0.9413 | - |
| 2.9825 | 340 | 0.7401 | - |
| 3.0 | 342 | - | 0.4185 |
| 3.0702 | 350 | 0.6514 | - |
| 3.1579 | 360 | 0.6765 | - |
| 3.2456 | 370 | 0.8422 | - |
| 3.3333 | 380 | 0.6532 | - |
| 3.4211 | 390 | 0.7121 | - |
| 3.5088 | 400 | 0.5739 | - |
| 3.5965 | 410 | 0.7838 | - |
| 3.6842 | 420 | 0.7554 | - |
| 3.7719 | 430 | 0.743 | - |
| 3.8596 | 440 | 0.5219 | - |
| 3.9474 | 450 | 0.8437 | - |
| 4.0 | 456 | - | 0.4187 |
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
@misc{kusupati2024matryoshka,
title={Matryoshka Representation Learning},
author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
year={2024},
eprint={2205.13147},
archivePrefix={arXiv},
primaryClass={cs.LG}
}
@misc{henderson2017efficient,
title={Efficient Natural Language Response Suggestion for Smart Reply},
author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
year={2017},
eprint={1705.00652},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
<!--
## Glossary
*Clearly define terms in order to be accessible across audiences.*
-->
<!--
## Model Card Authors
*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->
<!--
## Model Card Contact
*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->