Kiel-2-Poly

KielEmbed-Multilingual

A specialized multilingual text embedding model fine-tuned from sentence-transformers/paraphrase-multilingual-mpnet-base-v2.

This is a sentence-transformers model fine-tuned from sentence-transformers/paraphrase-multilingual-mpnet-base-v2. It maps sentences and paragraphs across multiple languages into a 768-dimensional dense vector space optimized for cross-lingual semantic similarity, retrieval, and clustering tasks.

Model Details

Model Description

• Model Type: Sentence Transformer / Multilingual Dense Embedding Backbone
• Base Model: sentence-transformers/paraphrase-multilingual-mpnet-base-v2
• Maximum Sequence Length: 128 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Supported Modality: Text (Multilingual)

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face Hub: kiel/KielEmbed-Multilingual

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'XLMRobertaModel'})
  (1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'mean', 'include_prompt': True})
)
UsageDirect Usage (Sentence Transformers)First, install the Sentence Transformers library:Bashpip install -U sentence-transformers
Then load your model and run inference:Pythonfrom sentence_transformers import SentenceTransformer

# Load your custom fine-tuned multilingual model from the Hugging Face Hub
model = SentenceTransformer("kiel/KielEmbed-Multilingual")

# Run inference
sentences = [
    '" Any decision on Charleroi will have huge implications for regional airports in France , " he said .',
    '" A bad decision on Charleroi would have huge implications for state-owned regional airports in France .',
    "He said the ferry 's crew will be interviewed and tested for drugs and alcohol .",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
Training DetailsTraining DatasetMultilingual Semantic CorpusSize: 10,000 training samplesColumns: text1, text2, and labelApproximate Token Statistics (First 100 samples):Text 1: Min: 15 tokens | Mean: 33.25 tokens | Max: 58 tokensText 2: Min: 16 tokens | Mean: 33.05 tokens | Max: 52 tokensLabel Distribution: Class 0 (~34.62%), Class 1 (~65.38%)Loss Function: CosineSimilarityLoss with parameters:JSON{
    "loss_fct": "torch.nn.modules.loss.MSELoss",
    "cos_score_transformation": "torch.nn.modules.linear.Identity"
}
Training HyperparametersPer Device Train Batch Size: 8Gradient Accumulation Steps: 4 (Effective batch size = 32)Learning Rate: 2e-05Number of Epochs: 1Warmup Steps: 0.1Mixed Precision: FP16 EnabledOptimizer: adamw_torch_fusedTraining LogsEpochStepTraining Loss0.165028.89790.3210024.76480.4815020.99410.6420026.49790.8025026.00710.9630026.2402Total Training Time: 6.5 minutesFramework VersionsPython: 3.13.15Sentence Transformers: 5.7.0Transformers: 5.16.1PyTorch: 2.11.0+cu128Accelerate: 1.14.0Datasets: 4.8.5Tokenizers: 0.23.1Additional ResourcesTraining and Finetuning Embedding Models with Sentence Transformers: End-to-end guide for training or fine-tuning Sentence Transformer models.Multilingual Embedding Models Guide: Learn more about cross-lingual retrieval and evaluation.CitationBibTeXCode snippet@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "[https://arxiv.org/abs/1908.10084](https://arxiv.org/abs/1908.10084)",
}