paraphrase-multilingual-MiniLM-L12-v2-helpfulness

SentenceTransformer based on sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2

This is a sentence-transformers model finetuned from sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 on the helpfulness-classification dataset. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 <!-- at revision ae06c001a2546bef168b9bf8f570ccb1a16aaa27 -->
• Maximum Sequence Length: 128 tokens
• Output Dimensionality: 384 tokens
• Similarity Function: Cosine Similarity
• Training Dataset:
  • helpfulness-classification
• Language: en

<!-- - **License:** Unknown -->

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    '<|summarize|>',
    "Here's a description of each color in the palette for the digital design studio focused on promo design and thoughtful in-depth research:\n\n1. #0d1925: This is a deep, dark blue color that conveys a sense of trust, stability, and expertise. It's a great choice for a design studio that wants to convey a professional and reliable image.\n\n2. #7391ab: This is a light blue color that suggests a sense of calmness, freshness, and innovation. It's a great choice for a design studio that wants to convey a modern and forward-thinking image.\n\n3. #ffc895: This is a bright, warm yellow color that suggests energy, optimism, and creativity. It's a great choice for a design studio that wants to convey a fun and innovative image.\n\n4. #f6f6f6: This is a light gray color that suggests neutrality, balance, and sophistication. It's a great choice for a design studio that wants to convey a refined and elegant image.\n\n5. #385a76: This is a dark green color that suggests growth, nature, and sustainability. It's a great choice for a design studio that wants to convey a environmentally-conscious and responsible image.\n\nOverall, this color palette is well-rounded and versatile, with colors that can be used in a variety of ways to convey different messages and emotions. The use of deep blues, light blues, bright yellows, and neutral grays makes it a great choice for a design studio that wants to convey a professional, modern, and innovative image.",
    'How about "Lingo Love"? This name plays on the idea of using language models to help with dating, suggesting that the app can help you find your perfect match by using sophisticated messaging features. The name also has a playful and romantic feel to it, which is fitting for a dating app.\n\nHere are a few other ideas to consider:\n\n1. "Chatterbox": This name suggests that the app is all about chatting and getting to know each other, which is a key aspect of dating. It also has a friendly and approachable feel to it.\n\n2. "Lingo Luck": This name combines the idea of using language models with the concept of luck in dating, suggesting that the app can help you find love by using advanced messaging features. It also has a playful and romantic feel to it.\n\n3. "Lingo Love Match": This name combines the idea of using language models with the concept of finding your perfect match, suggesting that the app can help you find your ideal partner through sophisticated messaging features. It also has a romantic and serious feel to it.\n\n4. "ChatterLover": This name combines the idea of chatting and getting to know each other with the concept of finding love, suggesting that the app can help you find your perfect match through engaging conversations. It also has a friendly and romantic feel to it.\n\n5. "Lingo Love Story": This name combines the idea of using language models with the concept of finding your perfect match and creating a love story, suggesting that the app can help you find your ideal partner and build a relationship through sophisticated messaging features. It also has a romantic and serious feel to it.\n\nOverall, a name for a dating app that uses a LLMs to create suggested messages should be catchy, memorable, and reflective of the app\'s core values and features. It should also have a tone that is appropriate for the target audience and the type of dating experience the app offers.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

<!-- ### Direct Usage (Transformers) <details><summary>Click to see the direct usage in Transformers</summary> </details> --> <!-- ### Downstream Usage (Sentence Transformers) You can finetune this model on your own dataset. <details><summary>Click to expand</summary> </details> --> <!-- ### Out-of-Scope Use *List how the model may foreseeably be misused and address what users ought not to do with the model.* -->

Evaluation

Metrics

Semantic Similarity

• Dataset: sts-dev
• Evaluated with <code>EmbeddingSimilarityEvaluator</code>

Semantic Similarity

• Dataset: sts-dev
• Evaluated with <code>EmbeddingSimilarityEvaluator</code>

Semantic Similarity

• Dataset: sts-dev
• Evaluated with <code>EmbeddingSimilarityEvaluator</code>

Semantic Similarity

• Dataset: sts-dev
• Evaluated with <code>EmbeddingSimilarityEvaluator</code>

<!-- ## Bias, Risks and Limitations *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* --> <!-- ### Recommendations *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* -->

Training Details

Training Dataset

helpfulness-classification

• Dataset: helpfulness-classification at 55a359a
• Size: 21,362 training samples
• Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	score
  type	string	string	float
  details	<ul><li>min: 4 tokens</li><li>mean: 44.54 tokens</li><li>max: 128 tokens</li></ul>	<ul><li>min: 3 tokens</li><li>mean: 119.41 tokens</li><li>max: 128 tokens</li></ul>	<ul><li>min: 0.0</li><li>mean: 0.72</li><li>max: 1.0</li></ul>

• Samples:

  sentence1	sentence2	score
  <code>c#</code>	<code>C# is a high-level, object-oriented programming language developed by Microsoft as part of its .NET initiative. It was created as a modern alternative to Java and supports a variety of programming paradigms, including imperative, functional, and event-driven. C# is primarily used for Windows application development, but it can also be used for web, mobile, and game development. The language is designed to be safe, secure, and efficient, and it provides developers with a rich set of libraries and tools for building robust and scalable applications. C# is also widely used in the game development industry, particularly in the development of games for the Xbox 360 and Xbox One consoles.</code>	<code>0.75</code>
  <code>c#</code>	<code>C# (pronounced "C sharp") is a modern, object-oriented programming language developed by Microsoft. It is widely used for building various types of applications, including web applications, desktop applications, mobile applications, and games. C# is similar to other programming languages such as Java and C++, and it is known for its simplicity and ease of use. C# is a powerful language that provides a rich set of libraries and frameworks that make it easy to build robust and scalable applications.<br><br>Here is a brief overview of some key features of C#:<br><br>1. Object-oriented: C# is an object-oriented language, which means it uses the concept of objects to represent real-world entities and their behavior.<br><br>2. Cross-platform: C# can be used to build applications for multiple platforms, including Windows, macOS, and Linux.<br><br>3. Strongly typed: C# is a strongly typed language, which means that variables must be declared with a specific type, and their type cannot be changed at runtime.<br><br>4. Event-driven: C# uses an event-driven programming model, which means that programs are built around the concept of events, such as user input or network activity.<br><br>5. Garbage-collected: C# has a garbage collector that automatically manages memory allocation and deallocation, making it easier to write memory-efficient and robust applications.<br><br>6. Community-driven: C# has a large and active community of developers, who contribute to the language and its libraries through open-source projects and other initiatives.<br><br>Overall, C# is a versatile and powerful programming language that is widely used for building a variety of applications.</code>	<code>1.0</code>
  <code>bacillus subtilus</code>	<code>Bacillus subtilis is a Gram-positive, rod-shaped bacterium commonly found in soil and water. It is a facultative anaerobe that can grow in the presence or absence of oxygen. B. subtilis is a model organism for studying bacterial cell biology, genetics, and physiology, and has many industrial and medical applications.<br><br>B. subtilis is a versatile organism with many interesting properties. It can form spores that allow it to survive harsh environmental conditions, such as heat, cold, and dehydration. The spores can remain dormant for long periods of time and then germinate when conditions are favorable. B. subtilis can also produce a variety of enzymes, antibiotics, and other compounds that have industrial and medical applications.<br><br>B. subtilis has been used in the production of enzymes for industrial applications, such as in the food and beverage industry for the production of cheese, yogurt, and beer. It is also used in the production of antibiotics and other pharmaceutical compounds. B. subtilis is also a model organism for studying bacterial cell biology, genetics, and physiology, and has been used extensively in research to study the mechanisms of bacterial growth, development, and survival.<br><br>Overall, B. subtilis is a versatile and important organism with many industrial and medical applications. Its properties and behaviors make it a valuable tool for researchers and industry alike.</code>	<code>1.0</code>

• Loss: <code>CosineSimilarityLoss</code> with these parameters:

  {
      "loss_fct": "torch.nn.modules.loss.MSELoss"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 64
• warmup_ratio: 0.1
• fp16: True

All Hyperparameters

<details><summary>Click to expand</summary>

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 8
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 3
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: False
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional

</details>

Training Logs

<details><summary>Click to expand</summary> </details>

Framework Versions

• Python: 3.10.14
• Sentence Transformers: 3.2.1
• Transformers: 4.45.1
• PyTorch: 2.4.0
• Accelerate: 0.34.2
• Datasets: 3.0.1
• Tokenizers: 0.20.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

<!-- ## Glossary *Clearly define terms in order to be accessible across audiences.* --> <!-- ## Model Card Authors *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.* --> <!-- ## Model Card Contact *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.* -->