roberta_chinese_3L312_clue_tiny

clue/roberta_chinese_3L312_clue_tiny

roberta_chinese_3L312_clue_tiny is a machine learning model from clue. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers.

This model was trained on TPU and the details are as follows:

Downloads · 30 days

131

2% of all-time downloads

All-time downloads

Public

Repo size

76.2 MB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.bin25.6 MB · 50%

At a glance

Library: transformers
Model type: roberta
Access: Public
Created: Mar 2, 2022
Updated: May 20, 2021
SHA: 5c87eb26

Model card

From the Hugging Face model README

Introduction

This model was trained on TPU and the details are as follows:

Model

Model_name	params	size	Training_corpus	Vocab
`RoBERTa-tiny-clue` <br/>Super_small_model	7.5M	28.3M	CLUECorpus2020	CLUEVocab
`RoBERTa-tiny-pair` <br/>Super_small_sentence_pair_model	7.5M	28.3M	CLUECorpus2020	CLUEVocab
`RoBERTa-tiny3L768-clue` <br/>small_model	38M	110M	CLUECorpus2020	CLUEVocab
`RoBERTa-tiny3L312-clue` <br/>small_model	<7.5M	24M	CLUECorpus2020	CLUEVocab
`RoBERTa-large-clue` <br/> Large_model	290M	1.20G	CLUECorpus2020	CLUEVocab
`RoBERTa-large-pair` <br/>Large_sentence_pair_model	290M	1.20G	CLUECorpus2020	CLUEVocab

Usage

With the help ofHuggingface-Transformers 2.5.1, you could use these model as follows

tokenizer = BertTokenizer.from_pretrained("MODEL_NAME")
model = BertModel.from_pretrained("MODEL_NAME")

MODEL_NAME：

Model_NAME	MODEL_LINK
RoBERTa-tiny-clue	`clue/roberta_chinese_clue_tiny`
RoBERTa-tiny-pair	`clue/roberta_chinese_pair_tiny`
RoBERTa-tiny3L768-clue	`clue/roberta_chinese_3L768_clue_tiny`
RoBERTa-tiny3L312-clue	`clue/roberta_chinese_3L312_clue_tiny`
RoBERTa-large-clue	`clue/roberta_chinese_clue_large`
RoBERTa-large-pair	`clue/roberta_chinese_pair_large`

Details

Please read <a href='https://arxiv.org/pdf/2003.01355'>https://arxiv.org/pdf/2003.01355.

Please visit our repository: https://github.com/CLUEbenchmark/CLUEPretrainedModels.git

Library: transformers
Type: roberta
Languages: zh
Created: Mar 2, 2022
Updated: May 20, 2021