Downloads · 30 days
20
36% of all-time downloads
bert-mini-amharic-16k
rasyosef/bert-mini-amharic-16k
bert-mini-amharic-16k is a fill-mask model from rasyosef. Use it when you need the model to fill a missing word. It is set up for transformers.
This model has the same architecture as bert-mini and was pretrained from scratch using the Amharic subsets of the oscar and mc4 datasets, on a total of 165 Million tokens. It achieves the following results on the eva…
All-time downloads
56
Public
Parameters
7.6M
30.3 MB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors30.3 MB · 98%
At a glance
- Task
- Fill-Mask
- Library
- transformers
- Model type
- bert
- Access
- Public
- Created
- Mar 27, 2024
- Updated
- May 9, 2024
- SHA
- ba9907af
Try a prompt
Model card
From the Hugging Face model README
bert-mini-amharic-16k
This model has the same architecture as bert-mini and was pretrained from scratch using the Amharic subsets of the oscar and mc4 datasets, on a total of 165 Million tokens.
It achieves the following results on the evaluation set:
Loss: 2.59Perplexity: 13.33
Even though this model only has 7.5 Million parameters, its perplexity score is comparable to the 36x larger 279 Million parameter xlm-roberta-base model on the same Amharic evaluation set.
- Task
- Fill-Mask
- Library
- transformers
- Type
- bert
- Languages
- am
- Created
- Mar 27, 2024
- Updated
- May 9, 2024