Downloads · 30 days
707
3% of all-time downloads
bert-base-cantonese
indiejoseph/bert-base-cantonese
bert-base-cantonese is a fill-mask model from indiejoseph. Use it when you need the model to fill a missing word. It is set up for transformers. The card lists the license as cc-by-4.0.
should probably proofread and complete it, then remove this comment. --
All-time downloads
26.5K
Public
Parameters
103M
15.2 GB on disk
Likes
9
Public
Repo makeup
Click a slice to open those files.
.bin411 MB · 50%
At a glance
- Task
- Fill-Mask
- Library
- transformers
- License
- cc-by-4.0
- Model type
- bert
- Access
- Public
- Created
- Sep 14, 2023
- Updated
- Apr 13, 2024
- SHA
- 65442e1c
Try a prompt
Base models
Model card
From the Hugging Face model README
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
bert-base-cantonese
This model is a continue pre-train version of bert-base-chinese on Cantonese Common Crawl dataset with 198m tokens.
Model description
This model has extended 500 more Chinese characters which very common in Cantonese, such as 冧, 噉, 麪, 笪, 冚, 乸 etc.
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 24
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 8
- total_train_batch_size: 192
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 1.0
Training results
Framework versions
- Transformers 4.35.0.dev0
- Pytorch 2.1.1+cu121
- Datasets 2.14.6
- Tokenizers 0.14.1
- Task
- Fill-Mask
- Library
- transformers
- Type
- bert
- License
- cc-by-4.0
- Languages
- yue
- Created
- Sep 14, 2023
- Updated
- Apr 13, 2024