Downloads · 30 days
6
3% of all-time downloads
MinhPhuc0804/me5-256-kiem-tra-di-t1-v3.2
me5-256-kiem-tra-di-t1-v3.2 is a sentence similarity model from MinhPhuc0804. Use it when you need a score for how close two texts are. It is set up for sentence-transformers.
This is a sentence-transformers model finetuned from intfloat/multilingual-e5-large-instruct on the clef-me5-triplets-hard-negatives dataset. It maps sentences & paragraphs to a 1024-dimensional dense vector space and…
Downloads · 30 days
6
3% of all-time downloads
All-time downloads
177
Public
Parameters
560M
2.3 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors2.2 GB · 99%
From the Hugging Face model README
This is a sentence-transformers model finetuned from intfloat/multilingual-e5-large-instruct on the clef-me5-triplets-hard-negatives dataset. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for retrieval.
SentenceTransformer(
(0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'XLMRobertaModel'})
(1): Pooling({'embedding_dimension': 1024, 'pooling_mode': 'mean', 'include_prompt': True})
(2): Normalize({})
)
First install the Sentence Transformers library:
pip install -U sentence-transformers
Then you can load this model and run inference.
from sentence_transformers import SentenceTransformer
# Download from the 🤗 Hub
model = SentenceTransformer("MinhPhuc0804/me5-256-kiem-tra-di-t1-v3.2")
# Run inference
sentences = [
'query: ICYMI: Revamped clinical guidelines for #MECFS have been rolled out by the US ME/CFS Clinician Coalition. The rising tide of fresh cases underscores the pressing necessity for physicians to get a better grasp of #MECFS and #longcovid.',
'passage: title: Myalgic Encephalomyelitis/Chronic Fatigue Syndrome: Essentials of Diagnosis and Management\nabstract: Despite myalgic encephalomyelitis/chronic fatigue syndrome (ME/CFS) affecting millions of people worldwide, many clinicians lack the knowledge to appropriately diagnose or manage ME/CFS. Unfortunately, clinical guidance has been scarce, obsolete, or potentially harmful. Consequently, up to 91% of patients in the United States remain undiagnosed, and those diagnosed often receive inappropriate treatment. These problems are of increasing importance because after acute COVID-19, a significant percentage of people remain ill for many months with an illness similar to ME/CFS. In 2015, the US National Academy of Medicine published new evidence-based clinical diagnostic criteria that have been adopted by the US Centers for Disease Control and Prevention. Furthermore, the United States and other governments as well as major health care organizations have recently withdrawn graded exercise and cognitive-behavioral therapy as the treatment of choice for patients with ME/CFS.',
'passage: title: <scp>Multi‐disciplinary</scp> collaborative consensus guidance statement on the assessment and treatment of autonomic dysfunction in patients with <scp>post‐acute</scp> sequelae of <scp>SARS‐CoV</scp>‐2 infection (<scp>PASC</scp>)\nabstract: Although many people infected with severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) recover completely, others are left with long-lasting symptoms that persist for at least 4 weeks, a condition referred to by the National Institutes of Health (NIH) as post-acute sequelae of SARS-CoV-2 infection (PASC).1 The Centers for Disease Control and Prevention (CDC) defines post-COVID (coronavirus disease) conditions as the wide range of health consequences that are present for four or more weeks after infection with SARS-CoV-2,2 whereas the World Health Organization (WHO) refers to post-COVID condition as symptoms that persist beyond 12 weeks after an initial infection, last for at least 2 months,',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]
# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.7066, 0.5363],
# [0.7066, 1.0000, 0.4503],
# [0.5363, 0.4503, 1.0000]])
<!--
### Direct Usage (Transformers)
<details><summary>Click to see the direct usage in Transformers</summary>
</details>
-->
<!--
### Downstream Usage (Sentence Transformers)
You can finetune this model on your own dataset.
<details><summary>Click to expand</summary>
</details>
-->
<!--
### Out-of-Scope Use
*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->
CT26-dev-split| Metric | Value |
|---|---|
| cosine_accuracy@1 | 0.6739 |
| cosine_accuracy@3 | 0.8328 |
| cosine_accuracy@5 | 0.8733 |
| cosine_accuracy@10 | 0.9128 |
| cosine_precision@1 | 0.6739 |
| cosine_precision@3 | 0.2776 |
| cosine_precision@5 | 0.1747 |
| cosine_precision@10 | 0.0913 |
| cosine_recall@1 | 0.6739 |
| cosine_recall@3 | 0.8328 |
| cosine_recall@5 | 0.8733 |
| cosine_recall@10 | 0.9128 |
| cosine_ndcg@10 | 0.7977 |
| cosine_mrr@10 | 0.7603 |
| cosine_map@100 | 0.7624 |
Dataset: clef-me5-triplets-hard-negatives
Size: 18,280 training samples
Columns: <code>anchor</code>, <code>positive</code>, and <code>negative</code>
Approximate statistics based on the first 1000 samples:
| anchor | positive | negative | |
|---|---|---|---|
| type | string | string | string |
| details | <ul><li>min: 26 tokens</li><li>mean: 59.6 tokens</li><li>max: 112 tokens</li></ul> | <ul><li>min: 22 tokens</li><li>mean: 188.95 tokens</li><li>max: 256 tokens</li></ul> | <ul><li>min: 27 tokens</li><li>mean: 194.91 tokens</li><li>max: 256 tokens</li></ul> |
Samples:
| anchor | positive | negative |
|---|---|---|
| <code>query: Peer-reviewed scientific studies project universal mask use would spare nearly 130,000 U.S. lives by February. But Utah carries on with a baffling, lax patchwork of feeble mask rules. If elected, I’ll shield your family. #PetersonProtects #utpol</code> | <code>passage: 469,578–578,347) lives could be lost to COVID-19 across the United States by 28 February 2021.<br><br>title: Modeling COVID-19 scenarios for the United States<br>We find that achieving universal mask use (95% mask use in public) could be sufficient to ameliorate the worst effects of epidemic resurgences in many states. Universal mask use could save an additional 129,574 (85,284–170,867) lives from September 22, 2020 through the end of February 2021, or an additional 95,814 (60,731–133,077) lives assuming a lesser adoption of mask wearing (85%), when compared to the reference scenario.</code> | <code>passage: title: Universal Masking to Prevent SARS-CoV-2 Transmission—The Time Is Now<br>abstract: Xiaowen Wang, MD; Enrico G. Ferro, MD; Guohai Zhou, PhD; Dean Hashimoto, MD, JD; Deepak L. Bhatt, MD, MPH</code> |
| <code>query: @KathMLee1 Jab wrecks their innate immunity- dampening IL6 VD8 NK’s- leaving you exposed not only to Covid but other hidden viruses and cancers- IME</code> | <code>passage: -term transcriptional changes in immune cells after vaccination, as assessed by RNA sequencing.<br><br>title: The impact of BNT162b2 mRNA vaccine on adaptive and innate immune responses<br>Additionally, vaccination with BNT162b2 modulated innate immune responses as measured by the production of inflammatory cytokines when stimulated with various microbial stimuli other than SARS-CoV-2, including higher IL-1/IL-6 release and decreased production of IFN-α. Altogether, these data expand our knowledge regarding the overall immunological effects of this new class of vaccines and underline the need of additional studies to elucidate their effects on both innate and adaptive immune responses.</code> | <code>passage: the genetic modifications introduced by the vaccine are likely the source of these differential responses. |
title: Innate Immune Suppression by SARS-CoV-2 mRNA Vaccinations: The role of G-quadruplexes, exosomes and microRNAs In this paper, we present the evidence that vaccination, unlike natural infection, induces a profound impairment in type I interferon signaling, which has diverse adverse consequences to human health. We explain the mechanism by which immune cells release into the circulation large quantities of exosomes containing spike protein along with critical microRNAs that induce a signaling response in recipient cells at distant sites. We also identify potential profound disturbances in regulatory control of protein synthesis and cancer surveillance. These disturbances are shown to have a potentially direct causal link to neurodegenerative disease, myocarditis, immune thrombocytopenia, Bell’s palsy, liver disease, impaired adaptive immunity, increased tumorigenesis...</code> | | <code>query: Rebuilding of October #Arctic sea ice volume spanning over the past 100 years... [Side-by-side look between PIOMAS-20C and PIOMAS data sets now refreshed through October 2021. Model details available at</code> | <code>passage: title: Arctic Sea Ice Volume Variability over 1901–2010: A Model-Based Reconstruction abstract: Abstract PIOMAS-20C, an Arctic sea ice reconstruction for 1901–2010, is produced by forcing the Pan-Arctic Ice Ocean Modeling and Assimilation System (PIOMAS) with ERA-20C atmospheric data. ERA-20C performance over Arctic sea ice is assessed by comparisons with measurements and data from other reanalyses. ERA-20C performs similarly with respect to the annual cycle of downwelling radiation, air temperature, and wind speed compared to reanalyses with more extensive data assimilation such as ERA-Interim and MERRA. PIOMAS-20C sea ice thickness and volume are then compared with in situ and aircraft remote sensing observations for the period of ~1950–2010. Error statistics are similar to those for PIOMAS. We compare the magnitude and patterns of sea ice variability between the first half of the twentieth century (1901–40) and the more recent period (1980–2010), both marked by sea ice decl...</code> | <code>passage: title: Variability of Arctic Sea Ice Thickness Using PIOMAS and the CESM Large Ensemble<br>abstract: Abstract Because of limited high-quality satellite and in situ observations, less attention has been given to the trends in Arctic sea ice thickness and therefore sea ice volume than to the trends in sea ice extent. This study evaluates the spatial and temporal variability in Arctic sea ice thickness using the Pan-Arctic Ice Ocean Modeling and Assimilation System (PIOMAS). Additionally, the Community Earth System Model Large Ensemble Project (LENS) is used to quantify the forced response and internal variability in the model. A dipole spatial pattern of sea ice thickness variability is shown in both PIOMAS and LENS with opposite signs of polarity between the East Siberian Sea and near the Fram Strait. As future sea ice thins, this dipole structure of variability is reduced, and the largest interannual variability is found only along the northern Greenland coastline.</code> |
Loss: <code>MultipleNegativesRankingLoss</code> with these parameters:
{
"scale": 20.0,
"similarity_fct": "cos_sim",
"gather_across_devices": false,
"directions": [
"query_to_doc"
],
"partition_mode": "joint",
"hardness_mode": null,
"hardness_strength": 0.0
}
| anchor | positive | negative | |
|---|---|---|---|
| type | string | string | string |
| details | <ul><li>min: 25 tokens</li><li>mean: 58.92 tokens</li><li>max: 121 tokens</li></ul> | <ul><li>min: 30 tokens</li><li>mean: 191.84 tokens</li><li>max: 256 tokens</li></ul> | <ul><li>min: 20 tokens</li><li>mean: 197.65 tokens</li><li>max: 256 tokens</li></ul> |
| anchor | positive | negative |
|---|---|---|
| <code>query: Hot off the press via @CDCgov's @CDCMMWR Efficacy of #COVID19Vaccines in Staving off #SARSCoV2 Infection Among essential staff pre- and amid #B16172 (#DeltaVariant) dominance — Eight U.S. sites, December 2020 – August 2021</code> | <code>passage: title: Effectiveness of COVID-19 Vaccines in Preventing SARS-CoV-2 Infection Among Frontline Workers Before and During B.1.617.2 (Delta) Variant Predominance — Eight U.S. Locations, December 2020–August 2021 | |
| abstract: During December 14, 2020-April 10, 2021, data from the HEROES-RECOVER Cohorts,* a network of prospective cohorts among frontline workers, showed that the Pfizer-BioNTech and Moderna mRNA COVID-19 vaccines were approximately 90% effective in preventing symptomatic and asymptomatic infection with SARS-CoV-2, the virus that causes COVID-19, in real-world conditions (1,2). This report updates vaccine effectiveness (VE) estimates including all COVID-19 vaccines available through August 14, 2021, and examines whether VE differs for adults with increasing time since completion of all recommended vaccine doses. VE before and during SARS-CoV-2 B.1.617.2 (Delta) variant predominance, which coincided with an increase in reported COVID-19 vaccine breakthrough infections, wer...</code> | <code>passage: horts of health care personnel, first responders, and other essential and frontline workers* in eight U.S.<br><br>title: Interim Estimates of Vaccine Effectiveness of BNT162b2 and mRNA-1273 COVID-19 Vaccines in Preventing SARS-CoV-2 Infection Among Health Care Personnel, First Responders, and Other Essential and Frontline Workers — Eight U.S. Locations, December 2020–March 2021<br>locations during December 14, 2020-March 13, 2021, CDC routinely tested for SARS-CoV-2 infections every week regardless of symptom status and at the onset of symptoms consistent with COVID-19-associated illness.</code> | |
| <code>query: Un #lait d'allaitement azur ou vert? Ceci n'est pas une plaisanterie, cela provient d'une recherche post injection de 178 #dames qui tétinent. 3 dames après la 1ère injection (Moderna ou Pfizer) et 2 dames après la 2e💉. Mais les médias persister à prôner d'injecter...</code> | <code>passage: title: Maternal and child outcomes reported by breastfeeding women following mRNA COVID-19 vaccination | |
| abstract: Table of Contents Summary What’s known on this subject One published U.S. study with 31 breastfeeding women who were vaccinated with either of the two available mRNA COVID-19 vaccines found that 67% experienced any side effects after dose one and 61% after dose two of the vaccine. What this study adds More than 85% of 180 breastfeeding women who received an mRNA COVID-19 vaccine reported local or systemic symptoms, with higher frequency following the second dose. Few symptoms were reported in their breastfed children. No serious adverse events were noted. Contributors’ Statement Page Dr. Chambers and Ms. Bertrand designed the study and supervised the collection of data used in the study. Mr. Honerkamp-Smith performed the statistical analysis. All authors were involved in preparing the manuscript. All authors approved the final manuscript as submitted and agree to be...</code> | <code>passage: title: Codominant IgG and IgA expression with minimal vaccine mRNA in milk of BNT162b2 vaccinees<br>abstract: Lactating women can produce protective antibodies in their milk after vaccination, which has informed antenatal vaccination programs for diseases such as influenza and pertussis. However, whether SARS-CoV-2-specific antibodies are produced in human milk as a result of COVID-19 vaccination is still unclear. In this study, we show that lactating mothers who received the BNT162b2 vaccine secreted SARS-CoV-2-specific IgA and IgG antibodies into milk, with the most significant increase at 3-7 days post-dose 2. Virus-specific IgG titers were stable out to 4-6 weeks after dose 2. In contrast, SARS-CoV-2-specific IgA levels showed substantial decay. Vaccine mRNA was detected in few milk samples (maximum of 2 ng/ml), indicative of minimal transfer. Additionally, infants who consumed post-vaccination human milk had no reported adverse effects up to 28 days post-ingestion.</code> | |
| <code>query: 📰Les scientifiques tentent présentement d'évaluer l'effet des récifs coralliens fabriqués sur les poissons. À ce stade, les données à modeste envergure pour le vivaneau rouge et le baliste gris dans le golfe du Mexique sont prometteurs.</code> | <code>passage: title: Fine-scale movement of northern Gulf of Mexico red snapper and gray triggerfish estimated with three-dimensional acoustic telemetry<br>abstract: Abstract Red snapper and gray triggerfish are ecologically, economically, and culturally important reef fishes in the northern Gulf of Mexico (nGOM). Scientists and managers have sought to understand the effects of artificial reefs on reef fish ecology by focusing on fish residency and movement at artificial reefs with less attention paid to broader spatial and temporal patterns in reef fish movements among a seascape of artificial reefs and other natural habitats. We used novel large-scale (> 15 km 2 ) geopositioning acoustic telemetry arrays to track the 3-dimensional movements of tagged red snapper (n = 59) and gray triggerfish (n = 15) among multiple nGOM artificial reefs up to 333 days.</code> | <code>passage: .<br><br>title: A new species of<i>Bathynomus</i>Milne-Edwards, 1879 (Isopoda: Cirolanidae) from the southern Gulf of Mexico with a redescription of<i>Bathynomus jamesi</i>Kou, Chen and Li, 2017 from off Pratas Island, Taiwan<br>Bathynomus kensleyi, previously known from the South China Sea as well as the south-east Swain Reefs, Coral Sea, was primarily differentiated by the elongate and upturned pleotelson spines, but that character is now shown to also occur in mature Bathynomus jamesi. Two specimens from the Gulf of Mexico (obtained from the Enoshima Aquarium in Japan) were compared to species of Bathynomus from the western North Atlantic. Sequence data showed that one of two samples was not B. giganteus Milne-Edwards, 1879, as had been assumed, and it did not match any other species of Bathynomus. That specimen was collected off the Yucatán Peninsula and is morphologically distinct from both B.</code> |
{
"scale": 20.0,
"similarity_fct": "cos_sim",
"gather_across_devices": false,
"directions": [
"query_to_doc"
],
"partition_mode": "joint",
"hardness_mode": null,
"hardness_strength": 0.0
}
per_device_train_batch_size: 256per_device_eval_batch_size: 256learning_rate: 6.4e-05num_train_epochs: 20warmup_ratio: 0.1bf16: Truedataloader_num_workers: 8load_best_model_at_end: Trueoptim: adamw_8bitgradient_checkpointing: Truegradient_checkpointing_kwargs: {'use_reentrant': False}overwrite_output_dir: Falsedo_predict: Falseprediction_loss_only: Trueper_device_train_batch_size: 256per_device_eval_batch_size: 256per_gpu_train_batch_size: Noneper_gpu_eval_batch_size: Nonegradient_accumulation_steps: 1eval_accumulation_steps: Nonetorch_empty_cache_steps: Nonelearning_rate: 6.4e-05weight_decay: 0.0adam_beta1: 0.9adam_beta2: 0.999adam_epsilon: 1e-08max_grad_norm: 1.0num_train_epochs: 20max_steps: -1lr_scheduler_type: linearlr_scheduler_kwargs: {}warmup_ratio: 0.1warmup_steps: 0log_level: passivelog_level_replica: warninglog_on_each_node: Truelogging_nan_inf_filter: Truesave_safetensors: Truesave_on_each_node: Falsesave_only_model: Falserestore_callback_states_from_checkpoint: Falseno_cuda: Falseuse_cpu: Falseuse_mps_device: Falseseed: 42data_seed: Nonejit_mode_eval: Falseuse_ipex: Falsebf16: Truefp16: Falsefp16_opt_level: O1half_precision_backend: autobf16_full_eval: Falsefp16_full_eval: Falsetf32: Nonelocal_rank: 0ddp_backend: Nonetpu_num_cores: Nonetpu_metrics_debug: Falsedebug: []dataloader_drop_last: Falsedataloader_num_workers: 8dataloader_prefetch_factor: Nonepast_index: -1disable_tqdm: Falseremove_unused_columns: Truelabel_names: Noneload_best_model_at_end: Trueignore_data_skip: Falsefsdp: []fsdp_min_num_params: 0fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}fsdp_transformer_layer_cls_to_wrap: Noneaccelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}parallelism_config: Nonedeepspeed: Nonelabel_smoothing_factor: 0.0optim: adamw_8bitoptim_args: Noneadafactor: Falsegroup_by_length: Falselength_column_name: lengthddp_find_unused_parameters: Noneddp_bucket_cap_mb: Noneddp_broadcast_buffers: Falsedataloader_pin_memory: Truedataloader_persistent_workers: Falseskip_memory_metrics: Trueuse_legacy_prediction_loop: Falsepush_to_hub: Falseresume_from_checkpoint: Nonehub_model_id: Nonehub_strategy: every_savehub_private_repo: Nonehub_always_push: Falsehub_revision: Nonegradient_checkpointing: Truegradient_checkpointing_kwargs: {'use_reentrant': False}include_inputs_for_metrics: Falseinclude_for_metrics: []eval_do_concat_batches: Truefp16_backend: autopush_to_hub_model_id: Nonepush_to_hub_organization: Nonemp_parameters:auto_find_batch_size: Falsefull_determinism: Falsetorchdynamo: Noneray_scope: lastddp_timeout: 1800torch_compile: Falsetorch_compile_backend: Nonetorch_compile_mode: Noneinclude_tokens_per_second: Falseinclude_num_input_tokens_seen: Falseneftune_noise_alpha: Noneoptim_target_modules: Nonebatch_eval_metrics: Falseeval_on_start: Falseuse_liger_kernel: Falseliger_kernel_config: Noneeval_use_gather_object: Falseaverage_tokens_across_devices: Falseprompts: Nonebatch_sampler: batch_samplermulti_dataset_batch_sampler: proportionalrouter_mapping: {}learning_rate_mapping: {}| Epoch | Step | Validation Loss | CT26-dev-split_cosine_ndcg@10 |
|---|---|---|---|
| 1.0 | 72 | 1.1980 | 0.8016 |
| 2.0 | 144 | 1.1289 | 0.7963 |
| 3.0 | 216 | 1.0538 | 0.8126 |
| 4.0 | 288 | 1.0634 | 0.8050 |
| 5.0 | 360 | 1.0911 | 0.7977 |
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
@misc{oord2019representationlearningcontrastivepredictive,
title={Representation Learning with Contrastive Predictive Coding},
author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
year={2019},
eprint={1807.03748},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/1807.03748},
}
<!--
## Glossary
*Clearly define terms in order to be accessible across audiences.*
-->
<!--
## Model Card Authors
*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->
<!--
## Model Card Contact
*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->