Downloads · 30 days
50
15% of all-time downloads
kokojake/modernbert-embed-base-fitness-health-matryoshka-5-epochs-25k
modernbert-embed-base-fitness-health-matryoshka-5-epochs-25k is a sentence similarity model from kokojake. Use it when you need a score for how close two texts are. It is set up for sentence-transformers. The card lists the license as apache-2.0.
This is a sentence-transformers model finetuned from nomic-ai/modernbert-embed-base on the json dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual simi…
Downloads · 30 days
50
15% of all-time downloads
All-time downloads
328
Public
Parameters
149M
596 MB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors596 MB · 99%
From the Hugging Face model README
This is a sentence-transformers model finetuned from nomic-ai/modernbert-embed-base on the json dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
SentenceTransformer(
(0): Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: ModernBertModel
(1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
(2): Normalize()
)
First install the Sentence Transformers library:
pip install -U sentence-transformers
Then you can load this model and run inference.
from sentence_transformers import SentenceTransformer
# Download from the 🤗 Hub
model = SentenceTransformer("kokojake/modernbert-embed-base-fitness-health-matryoshka-5-epochs-25k")
# Run inference
sentences = [
'CRONIN (Occupational therapist, USA); Diane DAMIANO (Physiotherapist, USA); Wouter de GROOTE (PRM physician, Belgium); Pamela ENDERBY (Speech and language therapist, United Kingdom); Darcy \nFEHLINGS (Developmental pediatrician/Clinician scientist, Canada); Charne FERIS (Occupational therapist, Namibia); Ferdiliza Dandah GARCIA (Speech pathologist and medical doctor, Philippines); Mohammad Mohinul ISLAM (Physiotherapist, Bangladesh); Heakyung KIM (PRM physician, USA); Pavlina PSYCHOULI (Occupational therapist, Greece); Mehdi RASSAFIANI (Occupational therapist, Islamic Republic of Iran); Gillian SALOOJEE (Physiotherapist, South Africa); Abena TANNOR (PRM physician/\nFamily medicine, Ghana).\nMembers of the peer review group An MIHEE (Physiotherapist, Republic of Korea); Uthman Olayiwola ANJORIN (Physiotherapist, Nigeria); Merce AVELLANET (PRM physician, Andorra); Marie BRIEN (Physiotherapist, Canada); Annemieke BUIZER',
'Developmental pediatrician research and studies',
'materials needed for plaster of Paris casts',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]
# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
<!--
### Direct Usage (Transformers)
<details><summary>Click to see the direct usage in Transformers</summary>
</details>
-->
<!--
### Downstream Usage (Sentence Transformers)
You can finetune this model on your own dataset.
<details><summary>Click to expand</summary>
</details>
-->
<!--
### Out-of-Scope Use
*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->
dim_768{
"truncate_dim": 768
}
| Metric | Value |
|---|---|
| cosine_accuracy@1 | 0.5371 |
| cosine_accuracy@3 | 0.5375 |
| cosine_accuracy@5 | 0.5375 |
| cosine_accuracy@10 | 0.5665 |
| cosine_precision@1 | 0.5371 |
| cosine_precision@3 | 0.5372 |
| cosine_precision@5 | 0.5372 |
| cosine_precision@10 | 0.489 |
| cosine_recall@1 | 0.033 |
| cosine_recall@3 | 0.0989 |
| cosine_recall@5 | 0.1649 |
| cosine_recall@10 | 0.2906 |
| cosine_ndcg@10 | 0.5035 |
| cosine_mrr@10 | 0.5421 |
| cosine_map@100 | 0.3249 |
dim_512{
"truncate_dim": 512
}
| Metric | Value |
|---|---|
| cosine_accuracy@1 | 0.5313 |
| cosine_accuracy@3 | 0.5313 |
| cosine_accuracy@5 | 0.5313 |
| cosine_accuracy@10 | 0.5618 |
| cosine_precision@1 | 0.5313 |
| cosine_precision@3 | 0.5313 |
| cosine_precision@5 | 0.5313 |
| cosine_precision@10 | 0.4845 |
| cosine_recall@1 | 0.0327 |
| cosine_recall@3 | 0.0981 |
| cosine_recall@5 | 0.1636 |
| cosine_recall@10 | 0.2886 |
| cosine_ndcg@10 | 0.4986 |
| cosine_mrr@10 | 0.5364 |
| cosine_map@100 | 0.3226 |
dim_256{
"truncate_dim": 256
}
| Metric | Value |
|---|---|
| cosine_accuracy@1 | 0.5162 |
| cosine_accuracy@3 | 0.5162 |
| cosine_accuracy@5 | 0.5162 |
| cosine_accuracy@10 | 0.5498 |
| cosine_precision@1 | 0.5162 |
| cosine_precision@3 | 0.5162 |
| cosine_precision@5 | 0.5162 |
| cosine_precision@10 | 0.4729 |
| cosine_recall@1 | 0.0318 |
| cosine_recall@3 | 0.0955 |
| cosine_recall@5 | 0.1592 |
| cosine_recall@10 | 0.2827 |
| cosine_ndcg@10 | 0.4859 |
| cosine_mrr@10 | 0.5218 |
| cosine_map@100 | 0.316 |
dim_128{
"truncate_dim": 128
}
| Metric | Value |
|---|---|
| cosine_accuracy@1 | 0.4927 |
| cosine_accuracy@3 | 0.4927 |
| cosine_accuracy@5 | 0.493 |
| cosine_accuracy@10 | 0.5193 |
| cosine_precision@1 | 0.4927 |
| cosine_precision@3 | 0.4927 |
| cosine_precision@5 | 0.4927 |
| cosine_precision@10 | 0.4484 |
| cosine_recall@1 | 0.0304 |
| cosine_recall@3 | 0.0912 |
| cosine_recall@5 | 0.152 |
| cosine_recall@10 | 0.2676 |
| cosine_ndcg@10 | 0.4617 |
| cosine_mrr@10 | 0.4971 |
| cosine_map@100 | 0.3035 |
dim_64{
"truncate_dim": 64
}
| Metric | Value |
|---|---|
| cosine_accuracy@1 | 0.4262 |
| cosine_accuracy@3 | 0.4262 |
| cosine_accuracy@5 | 0.4262 |
| cosine_accuracy@10 | 0.459 |
| cosine_precision@1 | 0.4262 |
| cosine_precision@3 | 0.4259 |
| cosine_precision@5 | 0.426 |
| cosine_precision@10 | 0.3932 |
| cosine_recall@1 | 0.0263 |
| cosine_recall@3 | 0.0787 |
| cosine_recall@5 | 0.1312 |
| cosine_recall@10 | 0.2349 |
| cosine_ndcg@10 | 0.4031 |
| cosine_mrr@10 | 0.4317 |
| cosine_map@100 | 0.2689 |
| positive | anchor | |
|---|---|---|
| type | string | string |
| details | <ul><li>min: 4 tokens</li><li>mean: 216.46 tokens</li><li>max: 412 tokens</li></ul> | <ul><li>min: 5 tokens</li><li>mean: 11.09 tokens</li><li>max: 38 tokens</li></ul> |
| positive | anchor |
|---|---|
| <code>5. Zeng CY, Zhang ZR, Tang ZM, Hua FZ. Benefits and mechanisms of exercise training for knee osteoarthritis.<br>Frontiers in Physiology. 2021;12. 6. Büssing A, Ostermann T, Lüdtke R, Michalsen A. Effects of yoga interventions on pain and pain-associated disability: a meta-analysis. J Pain. 2012;13(1):1-9. doi:10.1016/j.jpain.2011.10.001 7. Wren AA, Wright MA, Carson JW, Keefe FJ. Yoga for persistent pain: new findings and directions for an ancient practice. Pain. 2011;152(3):477-480. doi:10.1016/j.pain.2010.11.017 8. Lauche R, Hunter DJ, Adams J, Cramer H. Yoga for osteoarthritis: a systematic review and meta-analysis. Curr Rheumatol Rep. 2019;21(9):47. doi:10.1007/s11926-019-0846-5 9. Zhang Q, Young L, Li F. Network meta-analysis of various nonpharmacological interventions on pain relief in</code> | <code>yoga for persistent pain management</code> |
| <code>CRONIN (Occupational therapist, USA); Diane DAMIANO (Physiotherapist, USA); Wouter de GROOTE (PRM physician, Belgium); Pamela ENDERBY (Speech and language therapist, United Kingdom); Darcy <br>FEHLINGS (Developmental pediatrician/Clinician scientist, Canada); Charne FERIS (Occupational therapist, Namibia); Ferdiliza Dandah GARCIA (Speech pathologist and medical doctor, Philippines); Mohammad Mohinul ISLAM (Physiotherapist, Bangladesh); Heakyung KIM (PRM physician, USA); Pavlina PSYCHOULI (Occupational therapist, Greece); Mehdi RASSAFIANI (Occupational therapist, Islamic Republic of Iran); Gillian SALOOJEE (Physiotherapist, South Africa); Abena TANNOR (PRM physician/<br>Family medicine, Ghana).<br>Members of the peer review group An MIHEE (Physiotherapist, Republic of Korea); Uthman Olayiwola ANJORIN (Physiotherapist, Nigeria); Merce AVELLANET (PRM physician, Andorra); Marie BRIEN (Physiotherapist, Canada); Annemieke BUIZER</code> | <code>Developmental pediatrician research and studies</code> |
| <code>JAMA Network Open. 2025;8(4):e253698. doi:10.1001/jamanetworkopen.2025.3698<br>(Reprinted)<br>April 8, 2025</code> | <code>JAMA Network Open 2025 study on medical research</code> |
{
"loss": "MultipleNegativesRankingLoss",
"matryoshka_dims": [
768,
512,
256,
128,
64
],
"matryoshka_weights": [
1,
1,
1,
1,
1
],
"n_dims_per_step": -1
}
eval_strategy: epochper_device_train_batch_size: 32per_device_eval_batch_size: 16gradient_accumulation_steps: 16learning_rate: 2e-05num_train_epochs: 5lr_scheduler_type: cosinewarmup_ratio: 0.1bf16: Truetf32: Trueload_best_model_at_end: Trueoptim: adamw_torch_fusedbatch_sampler: no_duplicatesoverwrite_output_dir: Falsedo_predict: Falseeval_strategy: epochprediction_loss_only: Trueper_device_train_batch_size: 32per_device_eval_batch_size: 16per_gpu_train_batch_size: Noneper_gpu_eval_batch_size: Nonegradient_accumulation_steps: 16eval_accumulation_steps: Nonetorch_empty_cache_steps: Nonelearning_rate: 2e-05weight_decay: 0.0adam_beta1: 0.9adam_beta2: 0.999adam_epsilon: 1e-08max_grad_norm: 1.0num_train_epochs: 5max_steps: -1lr_scheduler_type: cosinelr_scheduler_kwargs: {}warmup_ratio: 0.1warmup_steps: 0log_level: passivelog_level_replica: warninglog_on_each_node: Truelogging_nan_inf_filter: Truesave_safetensors: Truesave_on_each_node: Falsesave_only_model: Falserestore_callback_states_from_checkpoint: Falseno_cuda: Falseuse_cpu: Falseuse_mps_device: Falseseed: 42data_seed: Nonejit_mode_eval: Falseuse_ipex: Falsebf16: Truefp16: Falsefp16_opt_level: O1half_precision_backend: autobf16_full_eval: Falsefp16_full_eval: Falsetf32: Truelocal_rank: 0ddp_backend: Nonetpu_num_cores: Nonetpu_metrics_debug: Falsedebug: []dataloader_drop_last: Falsedataloader_num_workers: 0dataloader_prefetch_factor: Nonepast_index: -1disable_tqdm: Falseremove_unused_columns: Truelabel_names: Noneload_best_model_at_end: Trueignore_data_skip: Falsefsdp: []fsdp_min_num_params: 0fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}tp_size: 0fsdp_transformer_layer_cls_to_wrap: Noneaccelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}deepspeed: Nonelabel_smoothing_factor: 0.0optim: adamw_torch_fusedoptim_args: Noneadafactor: Falsegroup_by_length: Falselength_column_name: lengthddp_find_unused_parameters: Noneddp_bucket_cap_mb: Noneddp_broadcast_buffers: Falsedataloader_pin_memory: Truedataloader_persistent_workers: Falseskip_memory_metrics: Trueuse_legacy_prediction_loop: Falsepush_to_hub: Falseresume_from_checkpoint: Nonehub_model_id: Nonehub_strategy: every_savehub_private_repo: Nonehub_always_push: Falsegradient_checkpointing: Falsegradient_checkpointing_kwargs: Noneinclude_inputs_for_metrics: Falseinclude_for_metrics: []eval_do_concat_batches: Truefp16_backend: autopush_to_hub_model_id: Nonepush_to_hub_organization: Nonemp_parameters:auto_find_batch_size: Falsefull_determinism: Falsetorchdynamo: Noneray_scope: lastddp_timeout: 1800torch_compile: Falsetorch_compile_backend: Nonetorch_compile_mode: Noneinclude_tokens_per_second: Falseinclude_num_input_tokens_seen: Falseneftune_noise_alpha: Noneoptim_target_modules: Nonebatch_eval_metrics: Falseeval_on_start: Falseuse_liger_kernel: Falseeval_use_gather_object: Falseaverage_tokens_across_devices: Falseprompts: Nonebatch_sampler: no_duplicatesmulti_dataset_batch_sampler: proportional| Epoch | Step | Training Loss | dim_768_cosine_ndcg@10 | dim_512_cosine_ndcg@10 | dim_256_cosine_ndcg@10 | dim_128_cosine_ndcg@10 | dim_64_cosine_ndcg@10 |
|---|---|---|---|---|---|---|---|
| 0.2198 | 10 | 54.8397 | - | - | - | - | - |
| 0.4396 | 20 | 26.5885 | - | - | - | - | - |
| 0.6593 | 30 | 20.9275 | - | - | - | - | - |
| 0.8791 | 40 | 17.6283 | - | - | - | - | - |
| 1.0 | 46 | - | 0.4713 | 0.4725 | 0.4562 | 0.4333 | 0.3646 |
| 1.0879 | 50 | 13.4942 | - | - | - | - | - |
| 1.3077 | 60 | 12.4011 | - | - | - | - | - |
| 1.5275 | 70 | 12.2302 | - | - | - | - | - |
| 1.7473 | 80 | 11.7666 | - | - | - | - | - |
| 1.9670 | 90 | 11.9032 | - | - | - | - | - |
| 2.0 | 92 | - | 0.4909 | 0.4865 | 0.4760 | 0.4501 | 0.3923 |
| 2.1758 | 100 | 9.4322 | - | - | - | - | - |
| 2.3956 | 110 | 9.692 | - | - | - | - | - |
| 2.6154 | 120 | 8.7793 | - | - | - | - | - |
| 2.8352 | 130 | 8.3124 | - | - | - | - | - |
| 3.0 | 138 | - | 0.5021 | 0.4964 | 0.4851 | 0.4572 | 0.3995 |
| 3.0440 | 140 | 7.258 | - | - | - | - | - |
| 3.2637 | 150 | 7.3585 | - | - | - | - | - |
| 3.4835 | 160 | 7.5519 | - | - | - | - | - |
| 3.7033 | 170 | 7.6819 | - | - | - | - | - |
| 3.9231 | 180 | 7.3011 | - | - | - | - | - |
| 4.0 | 184 | - | 0.5058 | 0.4973 | 0.4857 | 0.462 | 0.4015 |
| 4.1319 | 190 | 7.4137 | - | - | - | - | - |
| 4.3516 | 200 | 7.1914 | - | - | - | - | - |
| 4.5714 | 210 | 7.38 | - | - | - | - | - |
| 4.7912 | 220 | 7.3488 | - | - | - | - | - |
| 4.9011 | 225 | - | 0.5035 | 0.4986 | 0.4859 | 0.4617 | 0.4031 |
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
@misc{kusupati2024matryoshka,
title={Matryoshka Representation Learning},
author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
year={2024},
eprint={2205.13147},
archivePrefix={arXiv},
primaryClass={cs.LG}
}
@misc{henderson2017efficient,
title={Efficient Natural Language Response Suggestion for Smart Reply},
author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
year={2017},
eprint={1705.00652},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
<!--
## Glossary
*Clearly define terms in order to be accessible across audiences.*
-->
<!--
## Model Card Authors
*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->
<!--
## Model Card Contact
*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->