Downloads · 30 days
135
1% of all-time downloads
electra-base-vn
NlpHUST/electra-base-vn
electra-base-vn is a machine learning model from NlpHUST. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
ELECTRA is a method for self-supervised language representation learning. It can be used to pre-train transformer networks using relatively little compute. ELECTRA models are trained to distinguish "real" input tokens…
All-time downloads
15.3K
Public
Parameters
134M
1.1 GB on disk
Likes
6
Public
Repo makeup
Click a slice to open those files.
.bin535 MB · 50%
Parameter types
How the weights are stored.
F32134M · 100%
Model card
From the Hugging Face model README
ELECTRA
Introduction
ELECTRA is a method for self-supervised language representation learning. It can be used to pre-train transformer networks using relatively little compute. ELECTRA models are trained to distinguish "real" input tokens vs "fake" input tokens generated by another neural network, similar to the discriminator of a GAN. At small scale, ELECTRA achieves strong results even when trained on a single GPU. At large scale, ELECTRA achieves state-of-the-art results on the SQuAD 2.0 dataset.
Electra-base-vn is trained on more 148gb text with max length 512.
You can download tensorflow version at Electra base TF version
Contact information
For personal communication related to this project, please contact Nha Nguyen Van ([email protected]).
- Library
- transformers
- Type
- electra
- License
- apache-2.0
- Languages
- vi
- Created
- Mar 2, 2022
- Updated
- Dec 23, 2024