Downloads · 30 days
13
7% of all-time downloads
emilyyy04/xlm-roberta-base-burmese-nli-v3
xlm-roberta-base-burmese-nli-v3 is a text classification model from emilyyy04. Use it when you need a label for a piece of text. It is set up for transformers. The card lists the license as mit.
A Burmese NLI model fine-tuned from xlm-roberta-base, trained on a carefully cleaned and curated dataset, achieving 79% accuracy on the test set.
Downloads · 30 days
13
7% of all-time downloads
All-time downloads
189
Public
Parameters
278M
1.1 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors1.1 GB · 99%
From the Hugging Face model README
A Burmese NLI model fine-tuned from xlm-roberta-base, trained on a carefully cleaned and curated dataset, achieving 79% accuracy on the test set.
This model predicts the relationship between a premise and a hypothesis as one of:
xlm-roberta-baseentailment, neutral, contradictionThe dataset consists of ~10k [10,443] samples across three classes:
| Label | Class | Count |
|---|---|---|
| 0 | Entailment | 3,608 |
| 1 | Neutral | 3,466 |
| 2 | Contradiction | 3,369 |
and the dataset is prepared from:
genre field is included intended for future zero-shot / cross-genre experiments later.Steps:
To prevent data leakage caused by shared premises:
Instead of random shuffling:
GroupShuffleSplit is used
Samples with the same premise always stay in the same split
Prevents:
The model was trained using Hugging Face Transformers with the following key configurations:
1.5e-516 (train & evaluation)0.020.142The best checkpoint was automatically loaded at the end of training using early stopping and F1-based model selection.
The model performance is evaluated using:
| Epoch | Train Loss | Val Loss | Accuracy | F1 |
|---|---|---|---|---|
| 1 | 0.7579 | 0.7919 | 0.6415 | 0.6134 |
| 2 | 0.6764 | 0.6207 | 0.7291 | 0.7265 |
| 3 | 0.5801 | 0.6861 | 0.7443 | 0.7462 |
| 4 | 0.4554 | 0.6415 | 0.7481 | 0.7488 |
| 5 | 0.3848 | 0.6434 | 0.7646 | 0.7646 |
| 6 | 0.3564 | 0.7296 | 0.7608 | 0.7607 |
Training stopped early at epoch 6 due to validation performance plateau and the best model saved is at the f1 score of 0.7646.
{
"eval_loss": 0.5780,
"eval_accuracy": 0.7877,
"eval_f1": 0.7876
}
Label order: entailment, neutral, contradiction
[[481 50 18]
[ 78 418 20]
[ 14 20 476]]
Rows represent true labels, columns represent predicted labels.
| Class | Precision | Recall | F1-score | Support |
|---|---|---|---|---|
| Entailment | 0.84 | 0.88 | 0.86 | 549 |
| Neutral | 0.86 | 0.81 | 0.83 | 516 |
| Contradiction | 0.93 | 0.93 | 0.93 | 510 |
| Accuracy | 0.87 | 1575 | ||
| Macro Avg | 0.87 | 0.87 | 0.87 | 1575 |
| Weighted Avg | 0.87 | 0.87 | 0.87 | 1575 |
You can use the model as follows:
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
model_name = "emilyyy04/xlm-roberta-base-burmese-nli"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)
premise = "ဟုတ်တယ်ဗျခင်ဗျားပြောတာမှန်တယ်အခုခေတ်မှာလူငယ်တွေကနည်းပညာကိုအရင်ကထက်ပိုပြီးကျွမ်းကျင်လာကြတော့အလုပ်အကိုင်အခွင့်အလမ်းသစ်တွေအများကြီးပေါ်လာတာပေါ့"
hypothesis = "နည်းပညာတိုးတက်လာတာကလူငယ်တွေအတွက်အခွင့်အလမ်းကောင်းတွေဖြစ်စေတယ်"
inputs = tokenizer(
premise,
hypothesis,
return_tensors="pt",
truncation=True,
padding=True,
max_length=128
)
outputs = model(**inputs)
predicted_class = torch.argmax(outputs.logits, dim=1).item()
label_map = {0: "entailment", 1: "neutral", 2: "contradiction"}
print("Predicted label:", label_map[predicted_class])
# conf
probs = torch.softmax(outputs.logits, dim=-1)[0]
print("Confidence:", {k: round(float(probs[i]), 3) for i, k in label_map.items()})
This project is part of our Semester-8 CS-502 Natural Language Processing course.
| Name | Role | GitHub |
|---|---|---|
| Yoon Thiri Aung | Leader | https://github.com/yoon-thiri04 |
| Soe Sett Lynn | Member | https://github.com/ssettlynn |
| Thura Aung | Member | - |
Genre-aware and zero-shot classification is planned but not yet implemented
Performance may vary for:
Future improvements: