Downloads · 30 days
234
10% of all-time downloads
Roformer-base-japanese
ganchengguang/Roformer-base-japanese
Roformer-base-japanese is a fill-mask model from ganchengguang. Use it when you need the model to fill a missing word. It is set up for transformers. The card lists the license as apache-2.0.
This is Roformer model pretrained on texts in the Japanese language.
All-time downloads
2.3K
Public
Parameters
68.2M
1.1 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.bin273 MB · 50%
Parameter types
How the weights are stored.
F3268.2M · 100%
Try a prompt
Model card
From the Hugging Face model README
This is Roformer model pretrained on texts in the Japanese language.
3.45GB wikipedia text
trained 125M step
use the BERT BPE tokenizer.
If you want to fine-tune model. Please use
from transformers import BertTokenizer, RoFormerModel
BertTokenizer.from_pretrained('Roformer-base-japanese')
RoFormerModel.from_pretrained('Roformer-base-japanese')
The accuracy in JGLUE-marc-ja-v1.0 binary sentiment classification 95.12%
Contribute by Yokohama Nationaly University Mori Lab
@article{su2021roformer, title={Roformer: Enhanced transformer with rotary position embedding}, author={Su, Jianlin and Lu, Yu and Pan, Shengfeng and Wen, Bo and Liu, Yunfeng}, journal={arXiv preprint arXiv:2104.09864}, year={2021} }
- Task
- Fill-Mask
- Library
- transformers
- Type
- roberta
- License
- apache-2.0
- Created
- Sep 4, 2022
- Updated
- Jul 20, 2024