Downloads · 30 days
76
26% of all-time downloads
eluve-inc/SEA-LION-ModernBERT-600M-multilingual-NLI-26lang-2mil7
SEA-LION-ModernBERT-600M-multilingual-NLI-26lang-2mil7 is a machine learning model from eluve-inc. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
SEA-LION ModernBERT 600M Multilingual NLI 26lang 2mil7 is a natural language inference (NLI) model based on the ModernBERT architecture. The model was originally the pre-trained aisingapore/SEA-LION-ModernBERT-600M mo…
Downloads · 30 days
76
26% of all-time downloads
All-time downloads
287
Public
Parameters
613M
1.3 GB on disk
Likes
3
Public
Click a slice to open those files.
.safetensors1.2 GB · 97%
From the Hugging Face model README
SEA-LION ModernBERT 600M Multilingual NLI 26lang 2mil7 is a natural language inference (NLI) model based on the ModernBERT architecture. The model was originally the pre-trained aisingapore/SEA-LION-ModernBERT-600M model, which is then fine-tuned on MoritzLaurer/multilingual-NLI-26lang-2mil7. The model is then evaluated zero-shot on IndoNLI.
Zero-shot accuracy on IndoNLI:
dev Acc. | test_lay Acc. | test_expert Acc. | |
|---|---|---|---|
IndoNLI | 84.30 | 82.05 | 77.11 |
from transformers import pipeline
model_name = "eluve-inc/SEA-LION-ModernBERT-600M-multilingual-NLI-26lang-2mil7"
clf = pipeline("text-classification", model=model_name, tokenizer=model_name, top_k=None, device="cuda:0")
premise = "Minggu lalu saya mulai batuk pilek, dan sekarang saya sudah sembuh."
hypothesis = "Pasien sudah tidak lagi mengalami gejala batuk pilek."
out = clf({"text": premise, "text_pair": hypothesis})
# [{'label': 'entailment', 'score': 0.8801426887512207},
# {'label': 'neutral', 'score': 0.08782942593097687},
# {'label': 'contradiction', 'score': 0.03202790021896362}]
| Model | #params | Arch. | Training/Validation data (text) |
|---|---|---|---|
SEA-LION-ModernBERT-600M-multilingual-NLI-26lang-2mil7 | 600M | ModernBERT | Multilingual NLI (26 languages) |
labels: entailment (0), neutral (1), contradiction (2)The following hyperparameters were used during training:
learning_rate: 2e-05train_batch_size: 256gradient_accumulation_steps: 1seed: 42optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08lr_scheduler_type: cosinewarmup_ratio: 0.1weight_decay: 0.01num_epochs: 3mixed_precision_training: bf16[1] Mahendra, R., Aji, A. F., Louvan, S., Rahman, F., & Vania, C. (2021, November). IndoNLI: A Natural Language Inference Dataset for Indonesian. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics.
[2] Warner, B., Chaffin, A., Clavié, B., Weller, O., Hallström, O., Taghadouini, S., Gallagher, A., Biswas, R., Ladhak, F., Aarsen, T., Cooper, N., Adams, G., Howard, J., & Poli, I. (2024). Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference. arXiv preprint arXiv:2412.13663.