Downloads · 30 days
10
0% of all-time downloads
toastynews/electra-hongkongese-base-discriminator
electra-hongkongese-base-discriminator is a machine learning model from toastynews. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
ELECTRA trained exclusively with data from Hong Kong. A signaficant amount of Hongkongese/Cantonese/Yue is included in the training data.
Downloads · 30 days
10
0% of all-time downloads
All-time downloads
10.5K
Public
Repo size
1.3 GB
Likes
2
Public
Click a slice to open those files.
.h5437 MB · 50%
From the Hugging Face model README
ELECTRA trained exclusively with data from Hong Kong. A signaficant amount of Hongkongese/Cantonese/Yue is included in the training data.
This model is an alternative to Chinese models. It may offer better performance for tasks catering to the langauge usage of Hong Kongers. Yue Wikipedia is used which is much smaller than Chinese Wikipedia; this model will lack the breath of knowledge compared to other Chinese models.
This is the base model trained from the official repo. Further finetuning will be needed for use on downstream tasks. Other model sizes are also available.
The training data consists of mostly news articles and blogs. There is probably a bias towards formal language usage.
The following is the list of data sources. Total characters is about 507M.
| Data | % |
|---|---|
| News Articles / Blogs | 58% |
| Yue Wikipedia / EVCHK | 18% |
| Restaurant Reviews | 12% |
| Forum Threads | 12% |
| Online Fiction | 1% |
The following is the distribution of different languages within the corpus.
| Language | % |
|---|---|
| Standard Chinese | 62% |
| Hongkongese | 30% |
| English | 8% |
Model was trained on a single TPUv3 from the official repo with the default parameters.
| Parameter | Value |
|---|---|
| Batch Size | 256 |
| Max Sequence Size | 512 |
| Vocab Size | 30000 |
Research supported with Cloud TPUs from Google's TensorFlow Research Cloud (TFRC)
Average evaluation task results over 10 runs. Comparison using the original repo model and code. Chinese models are available from Joint Laboratory of HIT and iFLYTEK Research (HFL)
| Model | DRCD (EM/F1) | openrice-senti | lihkg-cat | wordshk-sem |
|---|---|---|---|---|
| Chinese | 86.6 / 91.7 | 79.1 | 67.4 | 88.1 |
| Hongkongese | 83.0 / 89.6 | 81.5 | 70.0 | 90.1 |