Downloads · 30 days
57
41% of all-time downloads
GreenBed4725/bge-m3-arxiv-cs-retriever
bge-m3-arxiv-cs-retriever is a sentence similarity model from GreenBed4725. Use it when you need a score for how close two texts are. It is set up for sentence-transformers. The card lists the license as mit.
This is a sentence-transformers model finetuned from BAAI/bge-m3. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for retrieval.
Downloads · 30 days
57
41% of all-time downloads
All-time downloads
138
Public
Parameters
568M
2.3 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors2.3 GB · 99%
From the Hugging Face model README
This is a sentence-transformers model finetuned from BAAI/bge-m3. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for retrieval.
SentenceTransformer(
(0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'XLMRobertaModel'})
(1): Pooling({'embedding_dimension': 1024, 'pooling_mode': 'cls', 'include_prompt': True})
(2): Normalize({})
)
First install the Sentence Transformers library:
pip install -U sentence-transformers
Then you can load this model and run inference.
from sentence_transformers import SentenceTransformer
# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
'How can I rewrite text so it hides where the writer is from?',
'Mitigating Bias in Text Classification via Prompt-Based Text Transformation\nThe presence of specific linguistic signals particular to a certain sub-group can become highly salient to language models during training. In automated decision-making settings, this may lead to biased outcomes when models rely on cues that correlate with protected characteristics. We investigate whether prompting ChatGPT to rewrite text using simplification, neutralisation, localisation, and formalisation can reduce demographic signals while preserving meaning. Experimental results show a statistically significant drop in location classification accuracy across multiple models after transformation, suggesting reduced reliance on group-specific language. At the same time, sentiment analysis and rating prediction tasks confirm that the core meaning of the reviews remains greatly intact. These results suggest that prompt-based rewriting offers a practical and generalisable approach for mitigating bias in text classification.',
'Beyond Outlining: Heterogeneous Recursive Planning for Adaptive Long-form Writing with Language Models\nLong-form writing agents require flexible integration and interaction across information retrieval, reasoning, and composition. Current approaches rely on predefined workflows and rigid thinking patterns to generate outlines before writing, resulting in constrained adaptability during writing. In this paper we propose WriteHERE, a general agent framework that achieves human-like adaptive writing through recursive task decomposition and dynamic integration of three fundamental task types: retrieval, reasoning, and composition. Our methodology features: 1) a planning mechanism that interleaves recursive task decomposition and execution, eliminating artificial restrictions on writing workflow; and 2) integration of task types that facilitates heterogeneous task decomposition. Evaluations on both fiction writing and technical report generation show that our method consistently outperforms state-of-the-art approaches across all automatic evaluation metrics, demonstrating the effectiveness and broad applicability of our proposed framework. We have publicly released our code and prompts to facilitate further research.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]
# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.3995, 0.1831],
# [0.3995, 1.0000, 0.1697],
# [0.1831, 0.1697, 1.0000]])
<!--
### Direct Usage (Transformers)
<details><summary>Click to see the direct usage in Transformers</summary>
</details>
-->
<!--
### Downstream Usage (Sentence Transformers)
You can finetune this model on your own dataset.
<details><summary>Click to expand</summary>
</details>
-->
<!--
### Out-of-Scope Use
*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->
<!--
## Bias, Risks and Limitations
*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->
<!--
### Recommendations
*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->
| anchor | positive | negative_1 | negative_2 | negative_3 | negative_4 | negative_5 | negative_6 | |
|---|---|---|---|---|---|---|---|---|
| type | string | string | string | string | string | string | string | string |
| modality | text | text | text | text | text | text | text | text |
| details | <ul><li>min: 9 tokens</li><li>mean: 18.34 tokens</li><li>max: 28 tokens</li></ul> | <ul><li>min: 187 tokens</li><li>mean: 323.69 tokens</li><li>max: 477 tokens</li></ul> | <ul><li>min: 165 tokens</li><li>mean: 291.34 tokens</li><li>max: 474 tokens</li></ul> | <ul><li>min: 123 tokens</li><li>mean: 284.37 tokens</li><li>max: 483 tokens</li></ul> | <ul><li>min: 109 tokens</li><li>mean: 290.24 tokens</li><li>max: 490 tokens</li></ul> | <ul><li>min: 122 tokens</li><li>mean: 304.28 tokens</li><li>max: 512 tokens</li></ul> | <ul><li>min: 115 tokens</li><li>mean: 297.33 tokens</li><li>max: 461 tokens</li></ul> | <ul><li>min: 165 tokens</li><li>mean: 314.77 tokens</li><li>max: 478 tokens</li></ul> |
| anchor | positive | negative_1 | negative_2 | negative_3 | negative_4 | negative_5 | negative_6 |
|---|---|---|---|---|---|---|---|
| <code>연속으로 바뀌는 데이터에서 개인 정보 지키면서 모델 업데이트 잘 하는 방법 없을까?</code> | <code>Differentially Private Online Federated Learning with Correlated Noise<br>We introduce a novel differentially private algorithm for online federated<br>learning that employs temporally correlated noise to enhance utility while<br>ensuring privacy of continuously released models. To address challenges posed<br>by DP noise and local updates with streaming non-iid data, we develop a<br>perturbed iterate analysis to control the impact of the DP noise on the<br>utility. Moreover, we demonstrate how the drift errors from local updates can<br>be effectively managed under a quasi-strong convexity condition. Subject to an<br>$(\epsilon, \delta)$-DP budget, we establish a dynamic regret bound over the<br>entire time horizon, quantifying the impact of key parameters and the intensity<br>of changes in dynamic environments. Numerical experiments confirm the efficacy<br>of the proposed algorithm.</code> | <code>Consistent Updates for Scalable Microservices<br>Online services are commonly implemented with a scalable microservice architecture, where isomorphic workers process client requests, recording persistent state in a backend data store. To maintain service, modifications to service functionality must be made on the fly -- i.e., as the service continues to process client requests -- but doing so is challenging. The central difficulty is that of avoiding inconsistencies from mixed-mode operation, caused by workers of current and new versions interacting via the data store. Some update methods avoid mixed-mode altogether, but only at the cost of substantial inefficiency -- by doubling resources (memory and compute), or by halving throughput. The alternative is an uncontrolled ``rolling'' update, which runs the risk of serious service failures arising from inconsistent mixed-mode behavior.<br> Ideally, it should appear to every client that a service update takes effect atomically; this ensures th...</code> | <code>Consistent community detection in multi-layer networks with<br> heterogeneous differential privacy<br>As network data has become increasingly prevalent, a substantial amount of<br>attention has been paid to the privacy issue in publishing network data. One of<br>the critical challenges for data publishers is to preserve the topological<br>structures of the original network while protecting sensitive information. In<br>this paper, we propose a personalized edge flipping mechanism that allows data<br>publishers to protect edge information based on each node's privacy preference.<br>It can achieve differential privacy while preserving the community structure<br>under the multi-layer degree-corrected stochastic block model after<br>appropriately debiasing, and thus consistent community detection in the<br>privatized multi-layer networks is achievable. Theoretically, we establish the<br>consistency of community detection in the privatized multi-layer network and<br>show that better privacy protection of edges can be obtained fo...</code> | <code>Improved Generalization Guarantees in Restricted Data Models<br>Differential privacy is known to protect against threats to validity incurred<br>due to adaptive, or exploratory, data analysis -- even when the analyst<br>adversarially searches for a statistical estimate that diverges from the true<br>value of the quantity of interest on the underlying population. The cost of<br>this protection is the accuracy loss incurred by differential privacy. In this<br>work, inspired by standard models in the genomics literature, we consider data<br>models in which individuals are represented by a sequence of attributes with<br>the property that where distant attributes are only weakly correlated. We show<br>that, under this assumption, it is possible to "re-use" privacy budget on<br>different portions of the data, significantly improving accuracy without<br>increasing the risk of overfitting.</code> | <code>Protecting Time Series Data with Minimal Forecast Loss<br>Forecasting could be negatively impacted due to anonymization requirements in<br>data protection legislation. To measure the potential severity of this problem,<br>we derive theoretical bounds for the loss to forecasts from additive<br>exponential smoothing models using protected data. Following the guidelines of<br>anonymization from the General Data Protection Regulation (GDPR) and California<br>Consumer Privacy Act (CCPA), we develop the $k$-nearest Time Series ($k$-nTS)<br>Swapping and $k$-means Time Series ($k$-mTS) Shuffling methods to create<br>protected time series data that minimizes the loss to forecasts while<br>preventing a data intruder from detecting privacy issues. For efficient and<br>effective decision making, we formally model an integer programming problem for<br>a perfect matching for simultaneous data swapping in each cluster. We call it a<br>two-party data privacy framework since our optimization model includes the<br>utilities of a data provide...</code> | <code>Privacy and Accuracy-Aware AI/ML Model Deduplication<br>With the growing adoption of privacy-preserving machine learning algorithms,<br>such as Differentially Private Stochastic Gradient Descent (DP-SGD), training<br>or fine-tuning models on private datasets has become increasingly prevalent.<br>This shift has led to the need for models offering varying privacy guarantees<br>and utility levels to satisfy diverse user requirements. However, managing<br>numerous versions of large models introduces significant operational<br>challenges, including increased inference latency, higher resource consumption,<br>and elevated costs. Model deduplication is a technique widely used by many<br>model serving and database systems to support high-performance and low-cost<br>inference queries and model diagnosis queries. However, none of the existing<br>model deduplication works has considered privacy, leading to unbounded<br>aggregation of privacy costs for certain deduplicated models and inefficiencies<br>when applied to deduplicate DP-tra...</code> | <code>Dynamically Adjusting Case Reporting Policy to Maximize Privacy and<br> Utility in the Face of a Pandemic<br>Supporting public health research and the public's situational awareness<br>during a pandemic requires continuous dissemination of infectious disease<br>surveillance data. Legislation, such as the Health Insurance Portability and<br>Accountability Act of 1996 (HIPAA) and recent state-level regulations, permits<br>sharing de-identified person-level data; however, current de-identification<br>approaches are limited. namely, they are inefficient, relying on retrospective<br>disclosure risk assessments, and do not flex with changes in infection rates or<br>population demographics over time. In this paper, we introduce a framework to<br>dynamically adapt de-identification for near-real time sharing of person-level<br>surveillance data. The framework leverages a simulation mechanism, capable of<br>application at any geographic level, to forecast the re-identification risk of<br>sharing the data under a wide range of gener...</code> |
| <code>How can I keep user data private while updating models on constantly changing data streams?</code> | <code>Differentially Private Online Federated Learning with Correlated Noise<br>We introduce a novel differentially private algorithm for online federated<br>learning that employs temporally correlated noise to enhance utility while<br>ensuring privacy of continuously released models. To address challenges posed<br>by DP noise and local updates with streaming non-iid data, we develop a<br>perturbed iterate analysis to control the impact of the DP noise on the<br>utility. Moreover, we demonstrate how the drift errors from local updates can<br>be effectively managed under a quasi-strong convexity condition. Subject to an<br>$(\epsilon, \delta)$-DP budget, we establish a dynamic regret bound over the<br>entire time horizon, quantifying the impact of key parameters and the intensity<br>of changes in dynamic environments. Numerical experiments confirm the efficacy<br>of the proposed algorithm.</code> | <code>Differentially Private Runtime Monitoring<br>Modern stream-based monitors collect detailed statistics of the runtime behavior of the system under observation. If the system runs in a privacy-sensitive context, this poses the risk of disclosing sensitive information. Differential privacy is the state-of-the-art approach for protecting sensitive information, however, integrating it into runtime monitoring is challenging: temporal operators can cause individual input values to influence multiple outputs over time, leading to repeated disclosure of private information. We propose an approach that automatically enforces differential privacy in stream-based monitoring specifications by analyzing temporal dependencies and injecting carefully calibrated noise into the specification. To preserve the utility of the outputs, we identify strategically chosen positions in the specification for noise injection and leverage tree-based mechanisms to mitigate the accuracy loss caused by noise injected int...</code> | <code>Continuous Evolution Pool: Taming Recurring Concept Drift in Online Time Series Forecasting<br>Recurring concept drift poses a dual challenge in online time series forecasting: mitigating catastrophic forgetting while adhering to strict privacy constraints that prevent retaining historical data. Existing approaches predominantly rely on parameter updates or experience replay, which inevitably suffer from knowledge overwriting or privacy risks. To address this, we propose the Continuous Evolution Pool (CEP), a privacy-preserving framework that maintains a dynamic pool of specialized forecasters. Instead of storing raw samples, CEP utilizes lightweight statistical genes to decouple concept identification from forecasting. Specifically, it employs a Retrieval mechanism to identify the nearest concept based on gene similarity, an Evolution strategy to spawn new forecasters upon detecting distribution shifts, and an Elimination policy to prune obsolete models under memory constraints. Experime...</code> | <code>Privacy-Preserving Nonlinear Observer Design Using Contraction Analysis<br>Real-time information processing applications such as those enabling a more intelligent infrastructure are increasingly focused on analyzing privacy-sensitive data obtained from individuals. To produce accurate statistics about the habits of a population of users of a system, this data might need to be processed through model-based estimators. Moreover, models of population dynamics, originating for example from epidemiology or the social sciences, are often necessarily nonlinear. Motivated by these trends, this paper presents an approach to design nonlinear privacy-preserving model-based observers, relying on additive input or output noise to give differential privacy guarantees to the individuals providing the input data. For the case of output perturbation, contraction analysis allows us to design convergent observers as well as set the level of privacy-preserving noise appropriately. Two examples illustrate the...</code> | <code>A Privacy-Preserving Outsourced Data Model in Cloud Environment<br>Nowadays, more and more machine learning applications, such as medical<br>diagnosis, online fraud detection, email spam filtering, etc., services are<br>provided by cloud computing. The cloud service provider collects the data from<br>the various owners to train or classify the machine learning system in the<br>cloud environment. However, multiple data owners may not entirely rely on the<br>cloud platform that a third party engages. Therefore, data security and privacy<br>problems are among the critical hindrances to using machine learning tools,<br>particularly with multiple data owners. In addition, unauthorized entities can<br>detect the statistical input data and infer the machine learning model<br>parameters. Therefore, a privacy-preserving model is proposed, which protects<br>the privacy of the data without compromising machine learning efficiency. In<br>order to protect the data of data owners, the epsilon-differential privacy is<br>used, and fog node...</code> | <code>Realistic Differentially-Private Transmission Power Flow Data Release<br>For the modeling, design and planning of future energy transmission networks,<br>it is vital for stakeholders to access faithful and useful power flow data,<br>while provably maintaining the privacy of business confidentiality of service<br>providers. This critical challenge has recently been somewhat addressed in [1].<br>This paper significantly extends this existing work. First, we reduce the<br>potential leakage information by proposing a fundamentally different<br>post-processing method, using public information of grid losses rather than<br>power dispatch, which achieve a higher level of privacy protection. Second, we<br>protect more sensitive parameters, i.e., branch shunt susceptance in addition<br>to series impedance (complete pi-model). This protects power flow data for the<br>transmission high-voltage networks, using differentially private<br>transformations that maintain the optimal power flow consistent with, and<br>faithful to, expected mo...</code> | <code>Privacy-Preserving Hierarchical Model-Distributed Inference<br>This paper focuses on designing a privacy-preserving Machine Learning (ML)<br>inference protocol for a hierarchical setup, where clients own/generate data,<br>model owners (cloud servers) have a pre-trained ML model, and edge servers<br>perform ML inference on clients' data using the cloud server's ML model. Our<br>goal is to speed up ML inference while providing privacy to both data and the<br>ML model. Our approach (i) uses model-distributed inference (model<br>parallelization) at the edge servers and (ii) reduces the amount of<br>communication to/from the cloud server. Our privacy-preserving hierarchical<br>model-distributed inference, privateMDI design uses additive secret sharing and<br>linearly homomorphic encryption to handle linear calculations in the ML<br>inference, and garbled circuit and a novel three-party oblivious transfer are<br>used to handle non-linear functions. privateMDI consists of offline and online<br>phases. We designed these phases in a...</code> |
| <code>계속 바뀌는 데이터로 휴대폰들끼리 모델을 만들면서 개인정보를 안전하게 지키는 방법?</code> | <code>Differentially Private Online Federated Learning with Correlated Noise<br>We introduce a novel differentially private algorithm for online federated<br>learning that employs temporally correlated noise to enhance utility while<br>ensuring privacy of continuously released models. To address challenges posed<br>by DP noise and local updates with streaming non-iid data, we develop a<br>perturbed iterate analysis to control the impact of the DP noise on the<br>utility. Moreover, we demonstrate how the drift errors from local updates can<br>be effectively managed under a quasi-strong convexity condition. Subject to an<br>$(\epsilon, \delta)$-DP budget, we establish a dynamic regret bound over the<br>entire time horizon, quantifying the impact of key parameters and the intensity<br>of changes in dynamic environments. Numerical experiments confirm the efficacy<br>of the proposed algorithm.</code> | <code>Version Age of Information with Contact Mobility in Gossip Networks<br>A gossip network is considered in which a source node updates its status while other nodes in the network aim at keeping track of it as it varies over time. Information gets disseminated by the source sending status updates to the nodes, and the nodes gossiping with each other. In addition, the nodes in the network are mobile, and can move to other nodes to get information, which we term contact mobility. The goal for the nodes is to remain as fresh as possible, i.e., to have the same information as the source's. To evaluate the freshness of information, we use the Version Age-of-Information (VAoI) metric, defined as the difference between the version of information available at a given node and that at the source. We analyze the effect of contact mobility on information dissemination in the gossip network using a Stochastic Hybrid System (SHS) framework for different topologies and mobility scalings with increasing nu...</code> | <code>One Password: An Encryption Scheme for Hiding Users' Register<br> Information<br>In recent years, the attack which leverages register information (e.g.<br>accounts and passwords) leaked from 3rd party applications to try other<br>applications is popular and serious. We call this attack "database collision".<br>Traditionally, people have to keep dozens of accounts and passwords for<br>different applications to prevent this attack. In this paper, we propose a<br>novel encryption scheme for hiding users' register information and preventing<br>this attack. Specifically, we first hash the register information using<br>existing safe hash function. Then the hash string is hidden, instead a<br>coefficient vector is stored for verification. Coefficient vectors of the same<br>register information are generated randomly for different applications. Hence,<br>the original information is hardly cracked by dictionary based attack or<br>database collision in practice. Using our encryption scheme, each user only<br>needs to keep one password ...</code> | <code>Secure Data Hiding for Contact Tracing<br>Contact tracing is an effective tool in controlling the spread of infectious<br>diseases such as COVID-19. It involves digital monitoring and recording of<br>physical proximity between people over time with a central and trusted<br>authority, so that when one user reports infection, it is possible to identify<br>all other users who have been in close proximity to that person during a<br>relevant time period in the past and alert them. One way to achieve this<br>involves recording on the server the locations, e.g. by reading and reporting<br>the GPS coordinates of a smartphone, of all users over time. Despite its<br>simplicity, privacy concerns have prevented widespread adoption of this method.<br>Technology that would enable the "hiding" of data could go a long way towards<br>alleviating privacy concerns and enable contact tracing at a very large scale.<br>In this article we describe a general method to hide data. By hiding, we mean<br>that instead of disclosing a data value x, we w...</code> | <code>Software implementation of the SNOW 3G Generator on iOS and Android<br> platforms<br>The standard for wireless communication of high-speed data in mobile phones<br>and data terminals, called LTE (Long-Term Evolution) and marketed as 4G/LTE, is<br>quickly being adopted worldwide. The security of this type of communication is<br>a crucial factor mainly due to its mobile and wireless nature. This work<br>includes a practical analysis of the SNOW 3G generator used to protect the<br>confidentiality and integrity in LTE communications. In particular, several<br>techniques to perform multiplications and LFSR operations have been studied and<br>implemented on both iOS and Android platforms. The evaluation of those<br>implementations led to some conclusions that could be used to improve the<br>efficiency of future implementations of the standard.</code> | <code>The Android Platform Security Model (2023)<br>Android is the most widely deployed end-user focused operating system. With<br>its growing set of use cases encompassing communication, navigation, media<br>consumption, entertainment, finance, health, and access to sensors, actuators,<br>cameras, or microphones, its underlying security model needs to address a host<br>of practical threats in a wide variety of scenarios while being useful to<br>non-security experts. To support this flexibility, Android's security model<br>must strike a difficult balance between security, privacy, and usability for<br>end users; provide assurances for app developers; and maintain system<br>performance under tight hardware constraints. This paper aims to both document<br>the assumed threat model and discuss its implications, with a focus on the<br>ecosystem context in which Android exists. We analyze how different security<br>measures in past and current Android implementations work together to mitigate<br>these threats, and, where there are speci...</code> | <code>Quotation-Based Data Retention Mechanism for Data Privacy in LLM-Empowered Network Services<br>The deployment of large language models (LLMs) for next-generation network optimization introduces novel data governance challenges. mobile network operators (MNOs) increasingly leverage generative artificial intelligence (AI) for traffic prediction, anomaly detection, and service personalization, requiring access to users' sensitive network usage data-including mobility patterns, traffic types, and location histories. Under the General Data Protection Regulation (GDPR), the California Consumer Privacy Act (CCPA), and similar regulations, users retain the right to withdraw consent and demand data deletion. However, extensive machine unlearning degrades model accuracy and incurs substantial computational costs, ultimately harming network performance for all users. We propose an iterative price discovery mechanism enabling MNOs to compensate users for data retention through sequential price quotat...</code> |
{
"scale": 20.0,
"similarity_fct": "cos_sim",
"gather_across_devices": false,
"directions": [
"query_to_doc"
],
"partition_mode": "joint",
"hardness_mode": null,
"hardness_strength": 0.0
}
num_train_epochs: 1.0learning_rate: 0.0001warmup_steps: 0.1bf16: Truegradient_checkpointing: Trueper_device_train_batch_size: 8num_train_epochs: 1.0max_steps: -1learning_rate: 0.0001lr_scheduler_type: linearlr_scheduler_kwargs: Nonewarmup_steps: 0.1optim: adamw_torch_fusedoptim_args: Noneweight_decay: 0.0adam_beta1: 0.9adam_beta2: 0.999adam_epsilon: 1e-08optim_target_modules: Nonegradient_accumulation_steps: 1average_tokens_across_devices: Truemax_grad_norm: 1.0label_smoothing_factor: 0.0bf16: Truefp16: Falsebf16_full_eval: Falsefp16_full_eval: Falsetf32: Nonegradient_checkpointing: Truegradient_checkpointing_kwargs: Nonetorch_compile: Falsetorch_compile_backend: Nonetorch_compile_mode: Noneuse_liger_kernel: Falseliger_kernel_config: Noneuse_cache: Falseneftune_noise_alpha: Nonetorch_empty_cache_steps: Noneauto_find_batch_size: Falselog_on_each_node: Truelogging_nan_inf_filter: Trueinclude_num_input_tokens_seen: nolog_level: passivelog_level_replica: warningdisable_tqdm: Falseproject: huggingfacetrackio_space_id: Nonetrackio_bucket_id: Nonetrackio_static_space_id: Noneper_device_eval_batch_size: 8prediction_loss_only: Trueeval_on_start: Falseeval_do_concat_batches: Trueeval_use_gather_object: Falseeval_accumulation_steps: Noneinclude_for_metrics: []batch_eval_metrics: Falsesave_only_model: Falsesave_on_each_node: Falseenable_jit_checkpoint: Falsepush_to_hub: Falsehub_private_repo: Nonehub_model_id: Nonehub_strategy: every_savehub_always_push: Falsehub_revision: Noneload_best_model_at_end: Falseignore_data_skip: Falserestore_callback_states_from_checkpoint: Falsefull_determinism: Falseseed: 42data_seed: Noneuse_cpu: Falseaccelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}parallelism_config: Nonedataloader_drop_last: Falsedataloader_num_workers: 0dataloader_pin_memory: Truedataloader_persistent_workers: Falsedataloader_prefetch_factor: Noneremove_unused_columns: Truelabel_names: Nonetrain_sampling_strategy: randomlength_column_name: lengthddp_find_unused_parameters: Noneddp_bucket_cap_mb: Noneddp_broadcast_buffers: Falseddp_static_graph: Noneddp_backend: Noneddp_timeout: 1800fsdp: Nonefsdp_config: Nonedeepspeed: Nonedebug: []skip_memory_metrics: Truedo_predict: Falseresume_from_checkpoint: Nonewarmup_ratio: Nonelocal_rank: -1prompts: Nonebatch_sampler: batch_samplermulti_dataset_batch_sampler: proportionalrouter_mapping: {}learning_rate_mapping: {}| Epoch | Step | Training Loss |
|---|---|---|
| 0.0121 | 50 | 2.4980 |
| 0.0242 | 100 | 2.3591 |
| 0.0364 | 150 | 1.6509 |
| 0.0485 | 200 | 1.2446 |
| 0.0606 | 250 | 1.2135 |
| 0.0727 | 300 | 1.0715 |
| 0.0848 | 350 | 1.1659 |
| 0.0970 | 400 | 1.0622 |
| 0.1091 | 450 | 1.0086 |
| 0.1212 | 500 | 0.9816 |
| 0.1333 | 550 | 0.9935 |
| 0.1455 | 600 | 0.9657 |
| 0.1576 | 650 | 0.8472 |
| 0.1697 | 700 | 0.8994 |
| 0.1818 | 750 | 0.8320 |
| 0.1939 | 800 | 0.8556 |
| 0.2061 | 850 | 0.8964 |
| 0.2182 | 900 | 0.7581 |
| 0.2303 | 950 | 0.7465 |
| 0.2424 | 1000 | 0.7048 |
| 0.2545 | 1050 | 0.7438 |
| 0.2667 | 1100 | 0.7254 |
| 0.2788 | 1150 | 0.6706 |
| 0.2909 | 1200 | 0.6496 |
| 0.3030 | 1250 | 0.7269 |
| 0.3152 | 1300 | 0.6338 |
| 0.3273 | 1350 | 0.7347 |
| 0.3394 | 1400 | 0.7235 |
| 0.3515 | 1450 | 0.5733 |
| 0.3636 | 1500 | 0.7029 |
| 0.3758 | 1550 | 0.7295 |
| 0.3879 | 1600 | 0.6137 |
| 0.4 | 1650 | 0.5917 |
| 0.4121 | 1700 | 0.6945 |
| 0.4242 | 1750 | 0.7340 |
| 0.4364 | 1800 | 0.5363 |
| 0.4485 | 1850 | 0.6380 |
| 0.4606 | 1900 | 0.6230 |
| 0.4727 | 1950 | 0.5763 |
| 0.4848 | 2000 | 0.6276 |
| 0.4970 | 2050 | 0.6330 |
| 0.5091 | 2100 | 0.5429 |
| 0.5212 | 2150 | 0.5862 |
| 0.5333 | 2200 | 0.5731 |
| 0.5455 | 2250 | 0.5107 |
| 0.5576 | 2300 | 0.6480 |
| 0.5697 | 2350 | 0.6043 |
| 0.5818 | 2400 | 0.5271 |
| 0.5939 | 2450 | 0.6115 |
| 0.6061 | 2500 | 0.5278 |
| 0.6182 | 2550 | 0.5130 |
| 0.6303 | 2600 | 0.5224 |
| 0.6424 | 2650 | 0.5711 |
| 0.6545 | 2700 | 0.5052 |
| 0.6667 | 2750 | 0.5806 |
| 0.6788 | 2800 | 0.4410 |
| 0.6909 | 2850 | 0.5809 |
| 0.7030 | 2900 | 0.5571 |
| 0.7152 | 2950 | 0.5721 |
| 0.7273 | 3000 | 0.5416 |
| 0.7394 | 3050 | 0.5355 |
| 0.7515 | 3100 | 0.4677 |
| 0.7636 | 3150 | 0.4543 |
| 0.7758 | 3200 | 0.4807 |
| 0.7879 | 3250 | 0.4724 |
| 0.8 | 3300 | 0.4492 |
| 0.8121 | 3350 | 0.5156 |
| 0.8242 | 3400 | 0.5054 |
| 0.8364 | 3450 | 0.5595 |
| 0.8485 | 3500 | 0.5290 |
| 0.8606 | 3550 | 0.4988 |
| 0.8727 | 3600 | 0.5503 |
| 0.8848 | 3650 | 0.4413 |
| 0.8970 | 3700 | 0.4577 |
| 0.9091 | 3750 | 0.5177 |
| 0.9212 | 3800 | 0.4534 |
| 0.9333 | 3850 | 0.4973 |
| 0.9455 | 3900 | 0.4160 |
| 0.9576 | 3950 | 0.4766 |
| 0.9697 | 4000 | 0.4285 |
| 0.9818 | 4050 | 0.4887 |
| 0.9939 | 4100 | 0.4308 |
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
@misc{oord2019representationlearningcontrastivepredictive,
title={Representation Learning with Contrastive Predictive Coding},
author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
year={2019},
eprint={1807.03748},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/1807.03748},
}
<!--
## Glossary
*Clearly define terms in order to be accessible across audiences.*
-->
<!--
## Model Card Authors
*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->
<!--
## Model Card Contact
*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->