Downloads · 30 days
17
0% of all-time downloads
CirBERTa-Chinese-Base
WENGSYX/CirBERTa-Chinese-Base
CirBERTa-Chinese-Base is a machine learning model from WENGSYX. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers.
All-time downloads
6.7K
Public
Repo size
1.8 GB
Likes
3
Public
Repo makeup
Click a slice to open those files.
.bin1.2 GB · 100%
At a glance
- Library
- transformers
- Model type
- deberta-v2
- Access
- Public
- Created
- Apr 14, 2022
- Updated
- Apr 14, 2022
- SHA
- 8e996262
Model card
From the Hugging Face model README
CirBERTa
Apply the Circular to the Pretraining Model
| 预训练模型 | 学习率 | batchsize | 设备 | 语料库 | 时间 | 优化器 |
|---|---|---|---|---|---|---|
| CirBERTa-Chinese-Base | 1e-5 | 256 | 10张3090+3张A100 | 200G | 2月 | AdamW |
使用通用语料（WuDao 200G) 进行无监督预训练
在多项中文理解任务上，CirBERTa-Base模型超过MacBERT-Chinese-Large/RoBERTa-Chinese-Large
加载与使用
依托于huggingface-transformers
from transformers import AutoTokenizer,AutoModel
tokenizer = AutoTokenizer.from_pretrained("WENGSYX/CirBERTa-Chinese-Base")
model = AutoModel.from_pretrained("WENGSYX/CirBERTa-Chinese-Base")
引用:
(暂时先引用这个，论文正在撰写...)
@misc{CirBERTa,
title={CirBERTa: Apply the Circular to the Pretraining Model},
author={Yixuan Weng},
howpublished={\url{https://github.com/WENGSYX/CirBERTa}},
year={2022}
}
- Library
- transformers
- Type
- deberta-v2
- Created
- Apr 14, 2022
- Updated
- Apr 14, 2022