Downloads · 30 days
39
18% of all-time downloads
FareedKhan/mixedbread-ai_mxbai-embed-large-v1_FareedKhan_prime_synthetic_data_2k_3_8
mixedbread-ai_mxbai-embed-large-v1_FareedKhan_prime_synthetic_data_2k_3_8 is a sentence similarity model from FareedKhan. Use it when you need a score for how close two texts are. It is set up for sentence-transformers.
This is a sentence-transformers model finetuned from mixedbread-ai/mxbai-embed-large-v1 on the json dataset. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual…
Downloads · 30 days
39
18% of all-time downloads
All-time downloads
212
Public
Parameters
335M
1.3 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors1.3 GB · 100%
From the Hugging Face model README
This is a sentence-transformers model finetuned from mixedbread-ai/mxbai-embed-large-v1 on the json dataset. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
SentenceTransformer(
(0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel
(1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)
First install the Sentence Transformers library:
pip install -U sentence-transformers
Then you can load this model and run inference.
from sentence_transformers import SentenceTransformer
# Download from the 🤗 Hub
model = SentenceTransformer("FareedKhan/mixedbread-ai_mxbai-embed-large-v1_FareedKhan_prime_synthetic_data_2k_3_8")
# Run inference
sentences = [
"\nCZEN 002 is a novel, investigational anti-fungal synthetic octapeptide, derived from alpha-Melanocyte-Stimulating Hormone (a-MSH), effective against Candida albicans, a major cause of vaginitis and candidiasis in individuals with compromised immune systems. This drug modulates inflammatory and immune responses while uniquely killing Candida albicans without direct membrane damage. Its mode of action targets an unidentified receptor in yeast, setting it apart from classical anti-fungal methods. CZEN 002's pharmacodynamics highlight its ability to effectively combat fungal infections without depending on membrane disruption. Importantly, it displays potential in affecting the MC1R gene/protein, a relation crucial for understanding its broader biological impacts and therapeutic utility. However, its status as investigational underscores the need for further research to fully elucidate its efficacy, safety, and target specificity.",
'Could you recommend a drug effective against Candida albicans, suitable for treating candidiasis with vaginitis, that also targets the MC1R gene/protein?',
'Can you find a DNA repair pathway that is hierarchically linked to Depyrimidination and contributes to genomic stability by playing a role in abasic site formation?',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]
# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
<!--
### Direct Usage (Transformers)
<details><summary>Click to see the direct usage in Transformers</summary>
</details>
-->
<!--
### Downstream Usage (Sentence Transformers)
You can finetune this model on your own dataset.
<details><summary>Click to expand</summary>
</details>
-->
<!--
### Out-of-Scope Use
*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->
dim_768| Metric | Value |
|---|---|
| cosine_accuracy@1 | 0.5099 |
| cosine_accuracy@3 | 0.5941 |
| cosine_accuracy@5 | 0.6089 |
| cosine_accuracy@10 | 0.6535 |
| cosine_precision@1 | 0.5099 |
| cosine_precision@3 | 0.198 |
| cosine_precision@5 | 0.1218 |
| cosine_precision@10 | 0.0653 |
| cosine_recall@1 | 0.5099 |
| cosine_recall@3 | 0.5941 |
| cosine_recall@5 | 0.6089 |
| cosine_recall@10 | 0.6535 |
| cosine_ndcg@10 | 0.5819 |
| cosine_mrr@10 | 0.5592 |
| cosine_map@100 | 0.5656 |
| positive | anchor | |
|---|---|---|
| type | string | string |
| details | <ul><li>min: 3 tokens</li><li>mean: 248.62 tokens</li><li>max: 512 tokens</li></ul> | <ul><li>min: 13 tokens</li><li>mean: 35.62 tokens</li><li>max: 106 tokens</li></ul> |
| positive | anchor |
|---|---|
| <code><br><br>Based on the provided information, the term "expression_absent" appears to relate to specific anatomical parts where a certain characteristic, property, or presence is not found or not applicable. In other words, it denotes parts of the body where a particular enzyme, protein, function, or feature is not expressed or does not occur.<br><br>The anatomy parts mentioned in "expression_absent" are:<br><br>1. metanephros - This refers to the developing renal system of the embryo, which eventually forms the functional kidney. It involves the metanephric glomerulus and other structures crucial for kidney development. If something is "expression_absent" in a metanephros, it could mean that kidney development or a specific function within this process is not occurring as expected.<br><br>2. decidua - This is the layer of tissue that develops in the lining of the uterus and becomes thickened during pregnancy. It plays a crucial role in protecting the developing embryo/fetus and supporting pregnancy until it can be delivered. "Expression_absent" in decidua would suggest that the tissue is missing or not forming as it should, possibly impacting pregnancy outcomes.<br><br>3. metanephric glomerulus - This is a specific structure within the developing kidney during embryonic development. It is part of the nephron, where the functional units of the kidney filter blood. If the “expression_absent” term applies to this, it would indicate a lack or abnormality in the development or function of these essential kidney structures, potentially leading to kidney disorders.<br><br>In general, the use of "expression_absent" in these anatomical contexts suggests a condition or development anomaly, which could be further evaluated through medical examination or research for understanding underlying causes and potential treatments.</code> | <code>Which genes act as common targets for Chromic nitrate and Chromic chloride?</code> |
| <code><br><br>Cushing's syndrome is a condition characterized by excessive production of cortisol, primarily due to overproduction by the adrenal gland, tumors of the pituitary gland, or ectopic production of cortisol. It is important to distinguish Cushing's syndrome from adrenal insufficiency, as they present with contrasting clinical features and require different management strategies.<br><br>### Key Differences in Clinical Features and Management<br><br>#### Cushing's Syndrome vs. Adrenal Insufficiency<br>- Presentation:<br> - Cushing's Syndrome: Caused by too much cortisol circulating in the body, it typically presents with features like weight gain, moon facies, purple striae, thin skin, central obesity, high blood pressure, osteoporosis, diabetes, and mood changes, among others.<br> - Adrenal Insufficiency: Characterized by</code> | <code>Which disease is linked to the AIRE gene and presents with sudden severe shock and bilateral adrenal hemorrhage, requiring immediate critical care due to its potentially fatal progression?</code> |
| <code><br>Atypical hemolytic-uremic syndrome (aHUS) with H factor anomaly, classified under MONDO as 'atypical hemolytic-uremic syndrome with H factor anomaly', is a disease falling under the classification of both aHUS susceptibility type 1 and atypical HUS. This condition is characterized by its association with human papillomavirus (HPV) infection. As a specific type of hemolytic uremic syndrome, aHUS with H factor anomaly is distinguished by its atypical presentation compared to the classic form. This genetic condition increases an individual's susceptibility to aHUS, a serious disorder that leads to the destruction of red blood cells and reduces kidney function. The presence of HPV further complicates the condition, highlighting the need for comprehensive treatment strategies and further research into the underlying mechanisms linking these two factors.</code> | <code>What disease falls under the classification of both atypical hemolytic uremic syndrome (aHUS) susceptibility type 1 and atypical HUS, and is also associated with human papillomavirus (HPV) infection?</code> |
{
"loss": "MultipleNegativesRankingLoss",
"matryoshka_dims": [
768
],
"matryoshka_weights": [
1
],
"n_dims_per_step": -1
}
eval_strategy: epochlearning_rate: 1e-05warmup_ratio: 0.1bf16: Truetf32: Falseload_best_model_at_end: Trueoverwrite_output_dir: Falsedo_predict: Falseeval_strategy: epochprediction_loss_only: Trueper_device_train_batch_size: 8per_device_eval_batch_size: 8per_gpu_train_batch_size: Noneper_gpu_eval_batch_size: Nonegradient_accumulation_steps: 1eval_accumulation_steps: Nonetorch_empty_cache_steps: Nonelearning_rate: 1e-05weight_decay: 0.0adam_beta1: 0.9adam_beta2: 0.999adam_epsilon: 1e-08max_grad_norm: 1.0num_train_epochs: 3max_steps: -1lr_scheduler_type: linearlr_scheduler_kwargs: {}warmup_ratio: 0.1warmup_steps: 0log_level: passivelog_level_replica: warninglog_on_each_node: Truelogging_nan_inf_filter: Truesave_safetensors: Truesave_on_each_node: Falsesave_only_model: Falserestore_callback_states_from_checkpoint: Falseno_cuda: Falseuse_cpu: Falseuse_mps_device: Falseseed: 42data_seed: Nonejit_mode_eval: Falseuse_ipex: Falsebf16: Truefp16: Falsefp16_opt_level: O1half_precision_backend: autobf16_full_eval: Falsefp16_full_eval: Falsetf32: Falselocal_rank: 0ddp_backend: Nonetpu_num_cores: Nonetpu_metrics_debug: Falsedebug: []dataloader_drop_last: Falsedataloader_num_workers: 0dataloader_prefetch_factor: Nonepast_index: -1disable_tqdm: Falseremove_unused_columns: Truelabel_names: Noneload_best_model_at_end: Trueignore_data_skip: Falsefsdp: []fsdp_min_num_params: 0fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}fsdp_transformer_layer_cls_to_wrap: Noneaccelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}deepspeed: Nonelabel_smoothing_factor: 0.0optim: adamw_torchoptim_args: Noneadafactor: Falsegroup_by_length: Falselength_column_name: lengthddp_find_unused_parameters: Noneddp_bucket_cap_mb: Noneddp_broadcast_buffers: Falsedataloader_pin_memory: Truedataloader_persistent_workers: Falseskip_memory_metrics: Trueuse_legacy_prediction_loop: Falsepush_to_hub: Falseresume_from_checkpoint: Nonehub_model_id: Nonehub_strategy: every_savehub_private_repo: Falsehub_always_push: Falsegradient_checkpointing: Falsegradient_checkpointing_kwargs: Noneinclude_inputs_for_metrics: Falseeval_do_concat_batches: Truefp16_backend: autopush_to_hub_model_id: Nonepush_to_hub_organization: Nonemp_parameters:auto_find_batch_size: Falsefull_determinism: Falsetorchdynamo: Noneray_scope: lastddp_timeout: 1800torch_compile: Falsetorch_compile_backend: Nonetorch_compile_mode: Nonedispatch_batches: Nonesplit_batches: Noneinclude_tokens_per_second: Falseinclude_num_input_tokens_seen: Falseneftune_noise_alpha: Noneoptim_target_modules: Nonebatch_eval_metrics: Falseeval_on_start: Falseuse_liger_kernel: Falseeval_use_gather_object: Falsebatch_sampler: batch_samplermulti_dataset_batch_sampler: proportional| Epoch | Step | Training Loss | dim_768_cosine_map@100 |
|---|---|---|---|
| 0 | 0 | - | 0.5078 |
| 0.0441 | 10 | 0.9027 | - |
| 0.0881 | 20 | 0.6956 | - |
| 0.1322 | 30 | 1.016 | - |
| 0.1762 | 40 | 0.667 | - |
| 0.2203 | 50 | 0.4914 | - |
| 0.2643 | 60 | 0.5978 | - |
| 0.3084 | 70 | 0.5328 | - |
| 0.3524 | 80 | 0.5801 | - |
| 0.3965 | 90 | 0.3767 | - |
| 0.4405 | 100 | 0.4763 | - |
| 0.4846 | 110 | 0.5214 | - |
| 0.5286 | 120 | 0.6835 | - |
| 0.5727 | 130 | 0.5613 | - |
| 0.6167 | 140 | 0.5564 | - |
| 0.6608 | 150 | 0.5356 | - |
| 0.7048 | 160 | 0.5279 | - |
| 0.7489 | 170 | 0.4445 | - |
| 0.7930 | 180 | 0.5346 | - |
| 0.8370 | 190 | 0.4747 | - |
| 0.8811 | 200 | 0.465 | - |
| 0.9251 | 210 | 0.4968 | - |
| 0.9692 | 220 | 0.4687 | - |
| 1.0 | 227 | - | 0.566 |
| 1.0132 | 230 | 0.286 | - |
| 1.0573 | 240 | 0.2611 | - |
| 1.1013 | 250 | 0.3128 | - |
| 1.1454 | 260 | 0.4078 | - |
| 1.1894 | 270 | 0.366 | - |
| 1.2335 | 280 | 0.2454 | - |
| 1.2775 | 290 | 0.2488 | - |
| 1.3216 | 300 | 0.1626 | - |
| 1.3656 | 310 | 0.1958 | - |
| 1.4097 | 320 | 0.2792 | - |
| 1.4537 | 330 | 0.242 | - |
| 1.4978 | 340 | 0.1897 | - |
| 1.5419 | 350 | 0.2574 | - |
| 1.5859 | 360 | 0.2348 | - |
| 1.6300 | 370 | 0.4793 | - |
| 1.6740 | 380 | 0.2859 | - |
| 1.7181 | 390 | 0.2393 | - |
| 1.7621 | 400 | 0.4538 | - |
| 1.8062 | 410 | 0.2759 | - |
| 1.8502 | 420 | 0.2357 | - |
| 1.8943 | 430 | 0.3837 | - |
| 1.9383 | 440 | 0.2114 | - |
| 1.9824 | 450 | 0.3309 | - |
| 2.0 | 454 | - | 0.5680 |
| 2.0264 | 460 | 0.2656 | - |
| 2.0705 | 470 | 0.2584 | - |
| 2.1145 | 480 | 0.1465 | - |
| 2.1586 | 490 | 0.1656 | - |
| 2.2026 | 500 | 0.1527 | - |
| 2.2467 | 510 | 0.2051 | - |
| 2.2907 | 520 | 0.2047 | - |
| 2.3348 | 530 | 0.1201 | - |
| 2.3789 | 540 | 0.118 | - |
| 2.4229 | 550 | 0.258 | - |
| 2.4670 | 560 | 0.2163 | - |
| 2.5110 | 570 | 0.1389 | - |
| 2.5551 | 580 | 0.2614 | - |
| 2.5991 | 590 | 0.1488 | - |
| 2.6432 | 600 | 0.151 | - |
| 2.6872 | 610 | 0.0874 | - |
| 2.7313 | 620 | 0.1677 | - |
| 2.7753 | 630 | 0.0759 | - |
| 2.8194 | 640 | 0.1279 | - |
| 2.8634 | 650 | 0.2564 | - |
| 2.9075 | 660 | 0.1805 | - |
| 2.9515 | 670 | 0.1509 | - |
| 2.9956 | 680 | 0.1402 | - |
| 3.0 | 681 | - | 0.5656 |
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
@misc{kusupati2024matryoshka,
title={Matryoshka Representation Learning},
author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
year={2024},
eprint={2205.13147},
archivePrefix={arXiv},
primaryClass={cs.LG}
}
@misc{henderson2017efficient,
title={Efficient Natural Language Response Suggestion for Smart Reply},
author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
year={2017},
eprint={1705.00652},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
<!--
## Glossary
*Clearly define terms in order to be accessible across audiences.*
-->
<!--
## Model Card Authors
*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->
<!--
## Model Card Contact
*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->