embedding_billingue_francais_bambara1

SentenceTransformer based on sentence-transformers/paraphrase-multilingual-mpnet-base-v2

This is a sentence-transformers model finetuned from sentence-transformers/paraphrase-multilingual-mpnet-base-v2. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/paraphrase-multilingual-mpnet-base-v2 <!-- at revision 84fccfe766bcfd679e39efefe4ebf45af190ad2d -->
• Maximum Sequence Length: 128 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

<!-- - **Training Dataset:** Unknown --> <!-- - **Language:** Unknown --> <!-- - **License:** Unknown -->

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: XLMRobertaModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("Fatoumataa/embedding_billingue_francais_bambara1")
# Run inference
sentences = [
    'expliquez le rôle du contrat social selon john locke.',
    "pour locke, le contrat social est un accord par lequel les individus sortent de l'état de nature pour protéger leurs droits naturels et établir un gouvernement limité. il fonde la société civile.",
    'protection des droits naturels,le contrat social sert à rendre le peuple entièrement soumis à un souverain absolu. (locke défend un gouvernement limité, pas une soumission absolue comme hobbes).',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

<!-- ### Direct Usage (Transformers) <details><summary>Click to see the direct usage in Transformers</summary> </details> --> <!-- ### Downstream Usage (Sentence Transformers) You can finetune this model on your own dataset. <details><summary>Click to expand</summary> </details> --> <!-- ### Out-of-Scope Use *List how the model may foreseeably be misused and address what users ought not to do with the model.* -->

Evaluation

Metrics

Information Retrieval

• Dataset: bambara_eval
• Evaluated with <code>InformationRetrievalEvaluator</code>

Information Retrieval

• Dataset: bambara_eval
• Evaluated with <code>InformationRetrievalEvaluator</code>

<!-- ## Bias, Risks and Limitations *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* --> <!-- ### Recommendations *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* -->

Training Details

Training Dataset

Unnamed Dataset

• Size: 3,225 training samples
• Columns: <code>anchor</code>, <code>positive</code>, <code>negative</code>, and <code>label</code>
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative	label
  type	string	string	string	float
  details	<ul><li>min: 6 tokens</li><li>mean: 20.85 tokens</li><li>max: 77 tokens</li></ul>	<ul><li>min: 7 tokens</li><li>mean: 59.01 tokens</li><li>max: 128 tokens</li></ul>	<ul><li>min: 11 tokens</li><li>mean: 51.72 tokens</li><li>max: 128 tokens</li></ul>	<ul><li>min: 1.0</li><li>mean: 1.0</li><li>max: 1.0</li></ul>

• Samples:

  anchor	positive	negative	label
  <code>quel instrument de musique traditionnel malien est une sorte de harpe-luth à 21 cordes, emblématique des griots ?</code>	<code>kora ye mali laadala fɔlifɛn ye min bɛ fɔ ni juru 21 ye, griotw bɛ baara kɛ ni a ye kosɛbɛ walasa ka fara u ka maanaw ni dɔnkiliw kan.</code>	<code>le balafon est la harpe-luth à 21 cordes. (confusion avec un xylophone africain).</code>	<code>1.0</code>
  <code>fɛn jumɛnw tun bɛ mali mansamara ka jago siratigɛ la?</code>	<code>les principaux produits échangés dans l'mansaya du mali étaient l'or, le sel, et les esclaves, complétés par des produits locaux comme le kola, le tissu et les céréales.</code>	<code>victor hugo tun ye faransi kanuya sɛbɛnnikɛla ŋana ye. 'les misérables' ye a ka baara tɔgɔba dɔ ye, gafe min bɛ sigida ka tɔɔrɔ jira.</code>	<code>1.0</code>
  <code>calcule le double de 9.</code>	<code>18. sabula 9 siɲɛ fila ye 18 ye.</code>	<code>24. erreur fréquente dans le calcul de double.</code>	<code>1.0</code>

• Loss: <code>MultipleNegativesRankingLoss</code> with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

Evaluation Dataset

Unnamed Dataset

• Size: 817 evaluation samples
• Columns: <code>anchor</code>, <code>positive</code>, <code>negative</code>, and <code>label</code>
• Approximate statistics based on the first 817 samples:

  	anchor	positive	negative	label
  type	string	string	string	float
  details	<ul><li>min: 5 tokens</li><li>mean: 20.91 tokens</li><li>max: 55 tokens</li></ul>	<ul><li>min: 8 tokens</li><li>mean: 58.13 tokens</li><li>max: 128 tokens</li></ul>	<ul><li>min: 11 tokens</li><li>mean: 51.09 tokens</li><li>max: 128 tokens</li></ul>	<ul><li>min: 1.0</li><li>mean: 1.0</li><li>max: 1.0</li></ul>

• Samples:

  anchor	positive	negative	label
  <code>kɔrɔfɔli bɛ dɛmɛ jumɛn don seko ni dɔnko faamuyali la ?</code>	<code>la critique d'art offre des perspectives d'analyse, éclaire le contexte, et propose des interprétations, aidant ainsi le public à approfondir sa compréhension et son appréciation des œuvres.</code>	<code>tifoyidi bɛ sɔrɔ kosɛbɛ ji walima dumuni dunni fɛ min nɔgɔlen don banakisɛ bɛ mɔgɔ min na, o ka banakɔtaa fɛ. banakunbɛnni ye ji minta, bolo saniya ani dumuni tobili ye.</code>	<code>1.0</code>
  <code>polifoni ye mun ye fɔli la?</code>	<code>la polyphonie est une technique de composition où plusieurs mélodies indépendantes sont jouées simultanément. elle crée une richesse harmonique.</code>	<code>mali la, waati jiginni bɛ kɛ sababu ye ka ja waati janw ni sanjiw kɛ minnu tɛ kɛ tuma bɛɛ. o bɛ suman tigɛ dɔgɔya ani ka sɛnɛ bila farati la. a ka gɛlɛn sɛnɛkɛlaw ma.</code>	<code>1.0</code>
  <code>quelles sont les habitudes à adopter pour améliorer la qualité de son sommeil ?</code>	<code>walasa ka sunɔgɔ ɲuman sɔrɔ, aw ka kan ka sunɔgɔcogo dɔ sigi sen kan tuma bɛɛ, ka aw yɛrɛ tanga ɛkranw ma sani aw ka sunɔgɔ, ka dibi ni lafiya sigida dilan, ani ka farikoloɲɛnajɛ kɛ tile fɛ. aw bɛ aw yɛrɛ tanga kafeyin ma wula fɛ, o fana nafa ka bon.</code>	<code>manger une grande quantité de nourriture juste avant de dormir. (cela peut perturber la digestion et le sommeil, c'est une mauvaise habitude, erreur conceptuelle).</code>	<code>1.0</code>

• Loss: <code>MultipleNegativesRankingLoss</code> with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 32
• learning_rate: 1e-05
• weight_decay: 0.01
• num_train_epochs: 8
• warmup_ratio: 0.1
• fp16: True
• load_best_model_at_end: True

All Hyperparameters

<details><summary>Click to expand</summary>

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 32
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 1e-05
• weight_decay: 0.01
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 8
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional

</details>

Training Logs

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.11.13
• Sentence Transformers: 4.1.0
• Transformers: 4.53.3
• PyTorch: 2.6.0+cu124
• Accelerate: 1.9.0
• Datasets: 4.0.0
• Tokenizers: 0.21.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

<!-- ## Glossary *Clearly define terms in order to be accessible across audiences.* --> <!-- ## Model Card Authors *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.* --> <!-- ## Model Card Contact *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.* -->