Downloads · 30 days
18
1% of all-time downloads
toastynews/electra-hongkongese-base-hkt-ws
electra-hongkongese-base-hkt-ws is a token classification model from toastynews. Use it when you need labels on individual words, such as names. It is set up for transformers. The card lists the license as apache-2.0.
should probably proofread and complete it, then remove this comment. --
Downloads · 30 days
18
1% of all-time downloads
All-time downloads
1.9K
Public
Repo size
1.3 GB
Likes
0
Public
Click a slice to open those files.
.h5434 MB · 50%
From the Hugging Face model README
This model is a fine-tuned version of toastynews/electra-hongkongese-base-discriminator on HKCanCor, CityU and AS for word segmentation.
Performs word segmentation on text from Hong Kong. There are two versions; hk trained with only text from Hong Kong, and hkt trained with text from Hong Kong and Taiwan. Each version have base and small model sizes.
Trained to handle both Hongkongese/Cantonese and Standard Chinese from Hong Kong. Text from other places and English do not work as well. The easiest way is to use with the CKIP Transformers libary.
HKCanCor, CityU and AS are converted to BI-encoded word segmentation dataset in Hugging Face format using code from finetune-ckip-transformers.
The following hyperparameters were used during training:
| dataset | token_f | token_p | token_r |
|---|---|---|---|
| ud yue_hk | 0.9404 | 0.9442 | 0.9367 |
| ud zh_hk | 0.9327 | 0.9404 | 0.9251 |
| hkcancor | 0.9875 | 0.9868 | 0.9883 |
| cityu | 0.9766 | 0.9756 | 0.9777 |
| as | 0.9652 | 0.9601 | 0.9704 |
Was trained on hkcancor. Reported for reference only.