Downloads · 30 days
11
3% of all-time downloads
mt5-Large-Romania
alexjerpelea/mt5-Large-Romania
mt5-Large-Romania is a text generation model from alexjerpelea. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
Fine-tuned google/mt5-large (1.2B params) on the Romanian Affective State Identification (ASI) benchmark, following the MASIVE (Deas et al. 2024) recipe.
All-time downloads
359
Public
Parameters
1.2B
4.9 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors4.9 GB · 100%
At a glance
- Task
- Text Generation
- License
- apache-2.0
- Model type
- mt5
- Access
- Public
- Created
- Apr 23, 2026
- Updated
- Apr 23, 2026
- SHA
- 0a5f5162
Base models
Model card
From the Hugging Face model README
mT5-Large fine-tuned on Romanian ASI
Fine-tuned google/mt5-large (1.2B params) on the Romanian Affective State Identification (ASI) benchmark, following the MASIVE (Deas et al. 2024) recipe.
Task
Given a Romanian text with an affective-state word masked as <extra_id_0>,
predict the masked word. Example:
Input: Mă simt foarte <extra_id_0> după ce am terminat cursul.
Target: <extra_id_0> mândru <extra_id_1>
Training
| Training data | 45,181 (text, masked_word) pairs extracted by pattern matching + LLM validation + human eval from Filmot, FULG, and 6 small Romanian datasets |
| Optimizer | Adafactor, lr 4e-4 linear decay, weight decay 0.01 |
| Batch size | 16, 3 epochs (8,472 steps) |
| Precision | bf16 |
| Hardware | 1× NVIDIA RTX A6000 48 GB |
| Wall clock | 2h 04m |
| Final val loss | 0.350 |
Evaluation (beam-5)
| Test | n | Acc@1 | Acc@3 | Acc@5 | MRR |
|---|---|---|---|---|---|
| val (seen vocab) | 2,658 | 57.7% | 77.5% | 83.2% | 0.68 |
| test (unseen vocab) | 5,315 | 0.79% | 1.52% | 2.20% | 0.013 |
| zero-shot mT5-large baseline, unseen | 5,315 | 14.5% | 18.7% | 18.8% | 0.17 |
On unseen vocabulary, Sim@1 (contextual BERT cosine) = 0.74: the fine-tune
predicts semantic near-synonyms of held-out emotion words (e.g. frică for
rușine).
Full details in the repo: https://github.com/Continual-Learning-Emotion-Group/Romanian_ASI/tree/mt5-finetune/pipeline/ft_mt5
Usage
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
tok = AutoTokenizer.from_pretrained("alexjerpelea/mt5-Large-Romania")
m = AutoModelForSeq2SeqLM.from_pretrained("alexjerpelea/mt5-Large-Romania")
text = "Am fost foarte <extra_id_0> după ce am terminat proiectul."
ids = tok(text, return_tensors="pt").input_ids
out = m.generate(ids, num_beams=5, num_return_sequences=5, max_new_tokens=10)
for o in out:
print(tok.decode(o, skip_special_tokens=False))
- Task
- Text Generation
- Type
- mt5
- License
- apache-2.0
- Languages
- ro
- Created
- Apr 23, 2026
- Updated
- Apr 23, 2026