Downloads · 30 days
19
0% of all-time downloads
toastynews/electra-hongkongese-base-hk-ws
electra-hongkongese-base-hk-ws is a token classification model from toastynews. Use it when you need labels on individual words, such as names. It is set up for transformers. The card lists the license as apache-2.0.
This model is a fine-tuned version of toastynews/electra-hongkongese-base-discriminator on HKCanCor and CityU for word segmentation.
Downloads · 30 days
19
0% of all-time downloads
All-time downloads
6.8K
Public
Parameters
108M
1.3 GB on disk
Likes
1
Public
Click a slice to open those files.
.h5434 MB · 33%
How the weights are stored.
F32108M · 100%
From the Hugging Face model README
This model is a fine-tuned version of toastynews/electra-hongkongese-base-discriminator on HKCanCor and CityU for word segmentation.
Performs word segmentation on text from Hong Kong. There are two versions; hk trained with only text from Hong Kong, and hkt trained with text from Hong Kong and Taiwan. Each version have base and small model sizes.
Trained to handle both Hongkongese/Cantonese and Standard Chinese from Hong Kong. Text from other places and English do not work as well. The easiest way is to use with the CKIP Transformers libary.
HKCanCor and CityU are converted to BI-encoded word segmentation dataset in Hugging Face format using code from finetune-ckip-transformers.
The following hyperparameters were used during training:
| dataset | token_f | token_p | token_r |
|---|---|---|---|
| ud yue_hk | 0.9462 | 0.9487 | 0.9437 |
| ud zh_hk | 0.9330 | 0.9402 | 0.9260 |
| hkcancor | 0.9895 | 0.9880 | 0.9909 |
| cityu | 0.9806 | 0.9793 | 0.9818 |
| as | 0.9225 | 0.9183 | 0.9267 |
Was trained on hkcancor. Reported for reference only.