Downloads · 30 days
15
1% of all-time downloads
attn-360M-30B
hazyresearch/attn-360M-30B
attn-360M-30B is a machine learning model from hazyresearch. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers.
This model is an Attention (Llama architecture) model pretrained on 30Bn tokens of the Pile corpus.
All-time downloads
1.6K
Public
Repo size
2.9 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin1.4 GB · 100%
At a glance
- Library
- transformers
- Model type
- gpt2
- Access
- Public
- Created
- Jun 7, 2024
- Updated
- Jun 7, 2024
- SHA
- 91cc02d0
Datasets
Model card
From the Hugging Face model README
Model Card
This model is an Attention (Llama architecture) model pretrained on 30Bn tokens of the Pile corpus.
Model Sources
The model implementation and training code that produced the model are provided here: https://github.com/HazyResearch/based
- Library
- transformers
- Type
- gpt2
- Languages
- en
- Created
- Jun 7, 2024
- Updated
- Jun 7, 2024