Downloads · 30 days
558
12% of all-time downloads
IndoELECTRA
ChristopherA08/IndoELECTRA
IndoELECTRA is a machine learning model from ChristopherA08. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers.
ELECTRA is a new method for self-supervised language representation learning. This repository contains the pre-trained Electra Base model (tensorflow 1.15.0) trained in a Large Indonesian corpus (~16GB of raw text | ~…
All-time downloads
4.7K
Public
Repo size
876 MB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.bin438 MB · 100%
At a glance
- Library
- transformers
- Model type
- electra
- Access
- Public
- Created
- Mar 2, 2022
- Updated
- Feb 4, 2021
- SHA
- ccebcb76
Datasets
Model card
From the Hugging Face model README
IndoBERT (Indonesian BERT Model)
Model description
ELECTRA is a new method for self-supervised language representation learning. This repository contains the pre-trained Electra Base model (tensorflow 1.15.0) trained in a Large Indonesian corpus (~16GB of raw text | ~2B indonesian words). IndoELECTRA is a pre-trained language model based on ELECTRA architecture for the Indonesian Language.
This model is base version which use electra-base config.
Intended uses & limitations
How to use
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("ChristopherA08/IndoELECTRA")
model = AutoModel.from_pretrained("ChristopherA08/IndoELECTRA")
tokenizer.encode("hai aku mau makan.")
[2, 8078, 1785, 2318, 1946, 18, 4]
Training procedure
The training of the model has been performed using Google's original Tensorflow code on eight core Google Cloud TPU v2. We used a Google Cloud Storage bucket, for persistent storage of training data and models.
- Library
- transformers
- Type
- electra
- Languages
- id
- Created
- Mar 2, 2022
- Updated
- Feb 4, 2021