Downloads · 30 days
67
1% of all-time downloads
bert-tiny-ita
mascIT/bert-tiny-ita
bert-tiny-ita is a fill-mask model from mascIT. Use it when you need the model to fill a missing word. It is set up for transformers.
bert-tiny-ita is an italian foundational model (based on bert-tiny) pretrained from scratch on 20k italian Wikipedia articles and on a wide collection of italian words and dictionary definitions. It uses 512 context w…
All-time downloads
5.1K
Public
Parameters
3.1M
36.7 MB on disk
Likes
4
Public
Repo makeup
Click a slice to open those files.
.bin12.3 MB · 49%
At a glance
- Task
- Fill-Mask
- Library
- transformers
- Model type
- bert
- Access
- Public
- Created
- Sep 24, 2023
- Updated
- Nov 27, 2023
- SHA
- f5e9b767
Try a prompt
Model card
From the Hugging Face model README
bert-tiny-ita is an italian foundational model (based on bert-tiny) pretrained from scratch on 20k italian Wikipedia articles and on a wide collection of italian words and dictionary definitions. It uses 512 context window size.
The project is still a work in progress, new versions will come with time.
Use it as a foundational model to be finetuned for specific italian tasks.
Training
- epochs: 250
- lr: 1e-5
- optim: AdamW
- weight_decay: 1e-4
Eval
- perplexity: 45 (it's a 12MB model!)
- Task
- Fill-Mask
- Library
- transformers
- Type
- bert
- Languages
- it
- Created
- Sep 24, 2023
- Updated
- Nov 27, 2023