all-MiniLM-Malaysian-Multi

SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2

This is a sentence-transformers model finetuned from sentence-transformers/all-MiniLM-L6-v2. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/all-MiniLM-L6-v2 <!-- at revision c9745ed1d9f207416be6d2e6f8de32d1f16199bf -->
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity

<!-- - **Training Dataset:** Unknown --> <!-- - **Language:** Unknown --> <!-- - **License:** Unknown -->

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'BertModel'})
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("along26/mpnet_manglish-sentence-transformer")
# Run inference
sentences = [
    'Why have critics accused Najib Razak of mishandling the economy and what evidence supports these claims?',
    'Mengapa pengkritik menuduh Najib Razak salah mengendalikan ekonomi dan bukti apa yang menyokong dakwaan ini?',
    "How does adding more reactant or product affect the equilibrium position of a chemical reaction? Explain using Le Chatelier's principle with at least three different examples.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000, -0.8678,  0.9288],
#         [-0.8678,  1.0000, -0.8804],
#         [ 0.9288, -0.8804,  1.0000]])

<!-- ### Direct Usage (Transformers) <details><summary>Click to see the direct usage in Transformers</summary> </details> --> <!-- ### Downstream Usage (Sentence Transformers) You can finetune this model on your own dataset. <details><summary>Click to expand</summary> </details> --> <!-- ### Out-of-Scope Use *List how the model may foreseeably be misused and address what users ought not to do with the model.* --> <!-- ## Bias, Risks and Limitations *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* --> <!-- ### Recommendations *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* -->

Training Details

Training Dataset

Unnamed Dataset

• Size: 139,404 training samples
• Columns: <code>sentence_0</code>, <code>sentence_1</code>, and <code>sentence_2</code>
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1	sentence_2
  type	string	string	string
  details	<ul><li>min: 12 tokens</li><li>mean: 221.26 tokens</li><li>max: 512 tokens</li></ul>	<ul><li>min: 21 tokens</li><li>mean: 262.59 tokens</li><li>max: 512 tokens</li></ul>	<ul><li>min: 13 tokens</li><li>mean: 241.29 tokens</li><li>max: 512 tokens</li></ul>

• Samples:

  sentence_0	sentence_1	sentence_2
  <code>Suppose there are 6 guests at a party. Among them, some are friends with each other and some are strangers. What is the minimum number of guests who must be friends with each other or at least strangers with each other, in order to guarantee that there are 3 guests who are all friends or 3 guests who are all strangers?</code>	<code>Eh, imagine got 6 people go party lah. Some of them kakis, some don't know each other one. How many people must be friends or at least don't know each other, so can confirm got 3 people all kakis or 3 people all don't know each other? Aiyoh, very headache leh!</code>	<code>Why is there still a lack of emphasis on science, technology, engineering, and mathematics (STEM) education in Malaysia?</code>
  <code>A photon at point A is entangled with a photon at point B. Using the quantum teleportation protocol, how can you transfer the quantum state of the photon at point A to point C, without physically moving the photon from point A to point C, given that a shared entangled pair of photons is available between point A and point C? Provide a step-by-step explanation of the procedure.</code>	<code>Foton pada titik A terikat dengan foton pada titik B. Menggunakan protokol teleportasi kuantum, bagaimana anda boleh memindahkan keadaan kuantum foton pada titik A ke titik C, tanpa memindahkan foton secara fizikal dari titik A ke titik C, diberikan bahawa sepasang foton terjerat yang dikongsi tersedia antara titik A dan titik C? Berikan penjelasan langkah demi langkah tentang prosedur.</code>	<code>Civil society groups and activists in Malaysia have expressed concern about the government's handling of the 1MDB scandal and the prosecution of those involved for several reasons. The 1MDB scandal involves allegations of massive corruption and money laundering at the state-owned investment fund, with billions of dollars allegedly misappropriated and used for personal gain.<br><br>Firstly, some civil society groups and activists have criticized the government's investigation and prosecution of those involved in the 1MDB scandal as being politically motivated and lacking in transparency. They argue that the government has selectively targeted certain individuals for prosecution, while others with close ties to the ruling party have been left untouched.<br><br>Secondly, there are concerns about the slow pace of the investigations and prosecutions, which have dragged on for years without any significant progress. Some civil society groups and activists have accused the government of deliberately slow...</code>
  <code>To solve this problem, we can use the generating functions method. Let's represent each number in the set {1, 2, 3, ..., 10} as a variable x raised to the power of that number. The generating function for this set is:<br><br>G(x) = x^1 + x^2 + x^3 + ... + x^10<br><br>Now, we want to find the coefficient of x^15 in the expansion of G(x)^3, as this will represent the number of ways to select 3 numbers from the set such that their sum is 15.<br><br>G(x)^3 = (x^1 + x^2 + x^3 + ... + x^10)^3<br><br>Expanding G(x)^3, we are looking for the coefficient of x^15. We can do this by finding the possible combinations of terms that multiply to x^15:<br><br>1. x^1 * x^5 * x^9<br>2. x^2 * x^4 * x^9<br>3. x^3 * x^4 * x^8<br>4. x^3 * x^5 * x^7<br>5. x^4 * x^5 * x^6<br><br>Now, let's count the number of ways each combination can be formed:<br><br>1. x^1 * x^5 * x^9: There is only 1 way to form this combination.<br>2. x^2 * x^4 * x^9: There is only 1 way to form this combination.<br>3. x^3 * x^4 * x^8: There is only 1 way to form this combination.<br>4. x^3 * x^5 * ...</code>	<code>Untuk menyelesaikan masalah ini, kita boleh menggunakan kaedah fungsi penjanaan. Mari kita wakili setiap nombor dalam set {1, 2, 3, ..., 10} sebagai pembolehubah x dinaikkan kepada kuasa nombor itu. Fungsi penjanaan untuk set ini ialah:<br><br>G(x) = x^1 + x^2 + x^3 + ... + x^10<br><br>Sekarang, kita ingin mencari pekali x^15 dalam pengembangan G(x)^3, kerana ini akan mewakili bilangan cara untuk memilih 3 nombor daripada set supaya jumlahnya ialah 15.<br><br>G(x)^3 = (x^1 + x^2 + x^3 + ... + x^10)^3<br><br>Mengembangkan G(x)^3, kita sedang mencari pekali bagi x^15. Kita boleh melakukan ini dengan mencari kemungkinan gabungan istilah yang didarab kepada x^15:<br><br>1. x^1 * x^5 * x^9<br>2. x^2 * x^4 * x^9<br>3. x^3 * x^4 * x^8<br>4. x^3 * x^5 * x^7<br>5. x^4 * x^5 * x^6<br><br>Sekarang, mari kita kira bilangan cara setiap gabungan boleh dibentuk:<br><br>1. x^1 * x^5 * x^9: Terdapat hanya 1 cara untuk membentuk gabungan ini.<br>2. x^2 * x^4 * x^9: Terdapat hanya 1 cara untuk membentuk gabungan ini.<br>3. x^3 * x^4 * x^8: Terdapat hanya 1 cara u...</code>	<code>Why does the Malaysian government still insist on implementing the controversial National Security Council Act, which gives the government excessive power to declare a security area and restrict civil liberties?</code>

• Loss: <code>TripletLoss</code> with these parameters:

  {
      "distance_metric": "TripletDistanceMetric.EUCLIDEAN",
      "triplet_margin": 5
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

<details><summary>Click to expand</summary>

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 3
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin
• router_mapping: {}
• learning_rate_mapping: {}

</details>

Training Logs

Framework Versions

• Python: 3.12.12
• Sentence Transformers: 5.1.2
• Transformers: 4.57.1
• PyTorch: 2.8.0+cu126
• Accelerate: 1.11.0
• Datasets: 4.0.0
• Tokenizers: 0.22.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

TripletLoss

@misc{hermans2017defense,
    title={In Defense of the Triplet Loss for Person Re-Identification},
    author={Alexander Hermans and Lucas Beyer and Bastian Leibe},
    year={2017},
    eprint={1703.07737},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}

<!-- ## Glossary *Clearly define terms in order to be accessible across audiences.* --> <!-- ## Model Card Authors *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.* --> <!-- ## Model Card Contact *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.* -->