Downloads · 30 days
11
35% of all-time downloads
electra_small_generator_uncased_en
keras/electra_small_generator_uncased_en
electra_small_generator_uncased_en is a feature extraction model from keras. Use it when you need embeddings to search or compare text. It is set up for keras-hub.
ELECTRA model is a pretraining approach for language models published by Google. Two transformer models are trained, a generator and a discriminator. The generator replaces tokens in a sequence and is trained as a mas…
All-time downloads
31
Public
Repo size
54.6 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.h554.6 MB · 100%
At a glance
- Task
- Feature Extraction
- Library
- keras-hub
- Access
- Public
- Created
- Oct 28, 2024
- Updated
- Jun 17, 2025
- SHA
- b93bdfc7
Model card
From the Hugging Face model README
Model Overview
ELECTRA model is a pretraining approach for language models published by Google. Two transformer models are trained, a generator and a discriminator. The generator replaces tokens in a sequence and is trained as a masked language model. The discriminator is trained to discern what tokens have been replaced. This method of pretraining is more efficient than comparable methods like masked language modeling, especially for small models.
Weights are released under the MIT License. Keras model code is released under the Apache 2 License.
Links
- ELECTRA API Documentation
- ELECTRA Model Paper
- KerasHub Beginner Guide
- KerasHub Model Publishing Guide
Installation
Keras and KerasHub can be installed with:
pip install -U -q keras-hub
pip install -U -q keras
Jax, TensorFlow, and Torch come preinstalled in Kaggle Notebooks. For instruction on installing them in another environment see the Keras Getting Started page.
Presets
The following model checkpoints are provided by the Keras team. Full code examples for each are available below.
| Preset name | Parameters | Description |
|---|---|---|
electra_small_discriminator_uncased_en | 13.55M | 12-layer small ELECTRA discriminator model. All inputs are lowercased. Trained on English Wikipedia + BooksCorpus. |
electra_small_generator_uncased_en | 13.55M | 12-layer small ELECTRA generator model. All inputs are lowercased. Trained on English Wikipedia + BooksCorpus. |
electra_base_discriminator_uncased_en | 109.48M | 12-layer base ELECTRA discriminator model. All inputs are lowercased. Trained on English Wikipedia + BooksCorpus. |
electra_base_generator_uncased_en | 33.58M | 12-layer base ELECTRA generator model. All inputs are lowercased. Trained on English Wikipedia + BooksCorpus. |
electra_large_discriminator_uncased_en | 335.14M | 24-layer large ELECTRA discriminator model. All inputs are lowercased. Trained on English Wikipedia + BooksCorpus. |
electra_large_generator_uncased_en | 51.07M | 24-layer large ELECTRA generator model. All inputs are lowercased. Trained on English Wikipedia + BooksCorpus. |
- Task
- Feature Extraction
- Library
- keras-hub
- Created
- Oct 28, 2024
- Updated
- Jun 17, 2025