Downloads · 30 days
62
0% of all-time downloads
mt5-xl-lm-adapt
DKYoon/mt5-xl-lm-adapt
mt5-xl-lm-adapt is a machine learning model from DKYoon. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
🤗 Language model initialized from mT5 and trained for an additional 100K steps on the Prefix LM objective using mC4 data.
All-time downloads
14.4K
Public
Parameters
3.7B
29.9 GB on disk
Likes
3
Public
Repo makeup
Click a slice to open those files.
.bin15 GB · 50%
At a glance
- Library
- transformers
- License
- apache-2.0
- Model type
- mt5
- Access
- Public
- Created
- Apr 13, 2023
- Updated
- Apr 22, 2024
- SHA
- 928510c9
Model card
From the Hugging Face model README
🤗 Language model initialized from mT5 and trained for an additional 100K steps on the Prefix LM objective using mC4 data.
Paper: Overcoming Catastrophic Forgetting in Zero-Shot Cross-Lingual Generation
Authors: Tu Vu, Aditya Barua, Brian Lester, Daniel Cer, Mohit Iyyer, Noah Constant
PyTorch port of the original Flax checkpoint at Google/T5X repository.
- Library
- transformers
- Type
- mt5
- License
- apache-2.0
- Created
- Apr 13, 2023
- Updated
- Apr 22, 2024