Downloads · 30 days
107
2% of all-time downloads
nanoT5-base
pnawrot/nanoT5-base
nanoT5-base is a machine learning model from pnawrot. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
Google's T5-v1.1-base pre-trained for 24 hours (80k steps / 256 batch size) on a single GPU in nanoT5 library for efficient pre-training.
All-time downloads
5.7K
Public
Repo size
5 GB
Likes
11
Public
Repo makeup
Click a slice to open those files.
.bin990 MB · 100%
At a glance
- Library
- transformers
- License
- apache-2.0
- Model type
- t5
- Access
- Public
- Created
- Apr 18, 2023
- Updated
- Apr 26, 2025
- SHA
- 282c5b7c
Datasets
Model card
From the Hugging Face model README
Google's T5-v1.1-base pre-trained for 24 hours (80k steps / 256 batch size) on a single GPU in nanoT5 library for efficient pre-training.
For more details about training check out the paper about this work.
For more details about the model refer to the original paper and original model weights.
It can be further fine-tuned on SuperNatural-Instructions dataset to achieve comparable performance to the same model pre-trained on 150x more data through "a combination of model and data parallelism [...] on slices of Cloud TPU Pods", each with 1024 TPUs.
- Library
- transformers
- Type
- t5
- License
- apache-2.0
- Languages
- en
- Created
- Apr 18, 2023
- Updated
- Apr 26, 2025