Downloads · 30 days
31
2% of all-time downloads
SlovenBERTcina
IMJONEZZ/SlovenBERTcina
SlovenBERTcina is a fill-mask model from IMJONEZZ. Use it when you need the model to fill a missing word. It is set up for transformers.
All-time downloads
1.4K
Public
Parameters
85.8M
2.7 GB on disk
Likes
3
Public
Repo makeup
Click a slice to open those files.
.pt1.4 GB · 50%
Parameter types
How the weights are stored.
F3285.8M · 100%
Try a prompt
Model card
From the Hugging Face model README
#Slovak RoBERTA Masked Language Model
###83Mil Parameters in small model
Medium and Large models coming soon!
RoBERTA pretrained tokenizer vocab and merges included.
##Training params:
-
Dataset: 8GB Slovak Monolingual dataset including ParaCrawl (monolingual), OSCAR, and several gigs of my own findings and cleaning.
-
Preprocessing: Tokenized with a pretrained ByteLevelBPETokenizer trained on the same dataset. Uncased, with s, pad, /s, unk, and mask special tokens.
-
Evaluation results:
- Mnoho ľudí tu MASK
- žije.
- žijú.
- je.
- trpí.
- Ako sa MASK
- máte
- máš
- má
- hovorí
- Plážová sezóna pod Zoborom patrí medzi MASK obdobia.
- ročné
- najkrajšie
- najobľúbenejšie
- najnáročnejšie
- Mnoho ľudí tu MASK
-
Limitations: The current model is fairly small, although it works very well. This model is meant to be finetuned on downstream tasks e.g. Part-of-Speech tagging, Question Answering, anything in GLUE or SUPERGLUE.
-
Credit: If you use this or any of my models in research or professional work, please credit me - Christopher Brousseau in said work.
- Task
- Fill-Mask
- Library
- transformers
- Type
- roberta
- Created
- Mar 2, 2022
- Updated
- Jul 7, 2023