Downloads · 30 days
195
3% of all-time downloads
distilbert_punctuator_zh
Qishuai/distilbert_punctuator_zh
distilbert_punctuator_zh is a token classification model from Qishuai. Use it when you need labels on individual words, such as names. It is set up for transformers.
The model is fine-tuned based on DistilBertForTokenClassification for adding punctuations to plain text (simplified Chinese). The model is fine-tuned based on distilled model bert-base-chinese.
All-time downloads
6.3K
Public
Parameters
59.2M
473 MB on disk
Likes
4
Public
Repo makeup
Click a slice to open those files.
.bin237 MB · 50%
At a glance
- Task
- Token Classification
- Library
- transformers
- Model type
- distilbert
- Access
- Public
- Created
- Mar 2, 2022
- Updated
- Mar 23, 2023
- SHA
- a203e53a
Try a prompt
Model card
From the Hugging Face model README
Punctuator for Simplified Chinese
The model is fine-tuned based on DistilBertForTokenClassification for adding punctuations to plain text (simplified Chinese). The model is fine-tuned based on distilled model bert-base-chinese.
Usage
from transformers import DistilBertForTokenClassification, DistilBertTokenizerFast
model = DistilBertForTokenClassification.from_pretrained("Qishuai/distilbert_punctuator_zh")
tokenizer = DistilBertTokenizerFast.from_pretrained("Qishuai/distilbert_punctuator_zh")
Model Overview
Training data
Combination of following three dataset:
- News articles of People's Daily 2014. Reference
Model Performance
- Validation with MSRA training dataset. Reference
- Metrics Report:
precision recall f1-score support C_COMMA 0.67 0.59 0.63 91566 C_DUNHAO 0.50 0.37 0.42 21013 C_EXLAMATIONMARK 0.23 0.06 0.09 399 C_PERIOD 0.84 0.99 0.91 44258 C_QUESTIONMARK 0.00 1.00 0.00 0 micro avg 0.71 0.67 0.69 157236 macro avg 0.45 0.60 0.41 157236 weighted avg 0.69 0.67 0.68 157236
- Task
- Token Classification
- Library
- transformers
- Type
- distilbert
- Created
- Mar 2, 2022
- Updated
- Mar 23, 2023