Downloads · 30 days
74
3% of all-time downloads
SRoBERTa-F
Andrija/SRoBERTa-F
SRoBERTa-F is a fill-mask model from Andrija. Use it when you need the model to fill a missing word. It is set up for transformers. The card lists the license as apache-2.0.
Trained on 43GB datasets that contain Croatian and Serbian language for one epochs (9.6 mil. steps, 3 epochs). Leipzig Corpus, OSCAR, srWac, hrWac, cc100-hr and cc100-sr datasets
All-time downloads
2.6K
Public
Repo size
647 MB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.bin322 MB · 98%
At a glance
- Task
- Fill-Mask
- Library
- transformers
- License
- apache-2.0
- Model type
- roberta
- Access
- Public
- Created
- Mar 2, 2022
- Updated
- Jan 3, 2023
- SHA
- 75e99c47
Try a prompt
Model card
From the Hugging Face model README
Transformer language model for Croatian and Serbian
Trained on 43GB datasets that contain Croatian and Serbian language for one epochs (9.6 mil. steps, 3 epochs). Leipzig Corpus, OSCAR, srWac, hrWac, cc100-hr and cc100-sr datasets
Validation number of exampels run for perplexity:1620487 sentences Perplexity:6.02 Start loss: 8.6 Final loss: 2.0 Thoughts: Model could be trained more, the training did not stagnate.
| Model | #params | Arch. | Training data |
|---|---|---|---|
Andrija/SRoBERTa-F | 80M | Fifth | Leipzig Corpus, OSCAR, srWac, hrWac, cc100-hr and cc100-sr (43 GB of text) |
- Task
- Fill-Mask
- Library
- transformers
- Type
- roberta
- License
- apache-2.0
- Languages
- hr, sr, multilingual
- Created
- Mar 2, 2022
- Updated
- Jan 3, 2023