Downloads · 30 days
53
7% of all-time downloads
bert-base-dv
jamescalam/bert-base-dv
bert-base-dv is a fill-mask model from jamescalam. Use it when you need the model to fill a missing word. It is set up for transformers. The card lists the license as apache-2.0.
Pretrained model on Dhivehi language using masked language modeling (MLM).
All-time downloads
725
Public
Repo size
1 GB
Likes
3
Public
Repo makeup
Click a slice to open those files.
.bin266 MB · 100%
At a glance
- Task
- Fill-Mask
- Library
- transformers
- License
- apache-2.0
- Model type
- bert
- Access
- Public
- Created
- Mar 2, 2022
- Updated
- Dec 13, 2021
- SHA
- bc4c0034
Try a prompt
Model card
From the Hugging Face model README
BERT base for Dhivehi
Pretrained model on Dhivehi language using masked language modeling (MLM).
Tokenizer
The WordPiece tokenizer uses several components:
- Normalization: lowercase and then NFKD unicode normalization.
- Pretokenization: splits by whitespace and punctuation.
- Postprocessing: single sentences are output in format
[CLS] sentence A [SEP]and pair sentences in format[CLS] sentence A [SEP] sentence B [SEP].
Training
Training was performed over 16M+ Dhivehi sentences/paragraphs put together by @ashraq. An Adam optimizer with weighted decay was used with following parameters:
- Learning rate: 1e-5
- Weight decay: 0.1
- Warmup steps: 10% of data
- Task
- Fill-Mask
- Library
- transformers
- Type
- bert
- License
- apache-2.0
- Languages
- dv
- Created
- Mar 2, 2022
- Updated
- Dec 13, 2021