Downloads · 30 days
2.1K
38% of all-time downloads
xlm-roberta-khmer-small
seanghay/xlm-roberta-khmer-small
xlm-roberta-khmer-small is a fill-mask model from seanghay. Use it when you need the model to fill a missing word. It is set up for transformers. The card lists the license as apache-2.0.
Training from scratch using Masked Language Modeling task on 5M Khmer sentences or 162M words or 578K unique words for 1M steps.
All-time downloads
5.5K
Public
Parameters
49.7M
199 MB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors199 MB · 100%
At a glance
- Task
- Fill-Mask
- Library
- transformers
- License
- apache-2.0
- Model type
- xlm-roberta
- Access
- Public
- Created
- Jul 22, 2024
- Updated
- Jul 22, 2024
- SHA
- 47dc1241
Model card
From the Hugging Face model README
XLMRoBERTa for Khmer Language
Training from scratch using Masked Language Modeling task on 5M Khmer sentences or 162M words or 578K unique words for 1M steps.
Training data is created by crawling publicly available publicly news sites and Wikipedia.
Why?
- xlm-roberta-base is big. (279M parameters, while this is only 49M parameters).
- xlm-roberta-base is not optimized for Khmer language.
- xlm-roberta-base Vocab size is bigger (250,002) and this model uses 8000 vocab size.
Usage
from transformers import pipeline
pipe = pipeline("fill-mask", "seanghay/xlm-roberta-khmer-small")
result = pipe("សួស្ដីកម្ពុ<mask>!")
print(result)
[
{"score": 0.8130345344543457, "token": 11, "token_str": "ជា", "sequence": "សួស្ដីកម្ពុជា!"},
{"score": 0.17512884736061096, "token": 160, "token_str": "ជ", "sequence": "សួស្ដីកម្ពុជ!"},
{"score": 0.0034702506382018328, "token": 143, "token_str": "ជា", "sequence": "សួស្ដីកម្ពុ ជា!"},
{"score": 0.00305828545242548, "token": 16, "token_str": "រ", "sequence": "សួស្ដីកម្ពុរ!"},
{"score": 0.0007526700501330197, "token": 133, "token_str": "គ", "sequence": "សួស្ដីកម្ពុគ!"},
]
License
Apache-2.0
Citation
No need. :)
- Task
- Fill-Mask
- Library
- transformers
- Type
- xlm-roberta
- License
- apache-2.0
- Languages
- km
- Created
- Jul 22, 2024
- Updated
- Jul 22, 2024