Downloads · 30 days
38
42% of all-time downloads
bert-base-dv
alakxender/bert-base-dv
bert-base-dv is a fill-mask model from alakxender. Use it when you need the model to fill a missing word. It is set up for transformers.
Pretrained model on Dhivehi language using masked language modeling (MLM).
All-time downloads
90
Public
Parameters
66.6M
1.1 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors266 MB · 100%
At a glance
- Task
- Fill-Mask
- Library
- transformers
- Model type
- bert
- Access
- Public
- Created
- Aug 16, 2024
- Updated
- Dec 11, 2024
- SHA
- 5c80e18c
Try a prompt
Datasets
Model card
From the Hugging Face model README
BERT base for Dhivehi
Pretrained model on Dhivehi language using masked language modeling (MLM).
Tokenizer
The WordPiece tokenizer uses several components:
- Normalization: lowercase and then NFKD unicode normalization.
- Pretokenization: splits by whitespace and punctuation.
- Postprocessing: single sentences are output in format
[CLS] sentence A [SEP]and pair sentences in format[CLS] sentence A [SEP] sentence B [SEP].
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 32
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 2
Framework versions
- Transformers 4.41.2
- Pytorch 2.3.1+cu121
- Datasets 2.19.2
- Tokenizers 0.19.1
- Task
- Fill-Mask
- Library
- transformers
- Type
- bert
- Languages
- dv
- Created
- Aug 16, 2024
- Updated
- Dec 11, 2024