Downloads · 30 days
36
5% of all-time downloads
gla-1B-100B
bailin28/gla-1B-100B
gla-1B-100B is a text generation model from bailin28. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
This checkpoint of the 1.3B GLA model used in the paper Gated Linear Attention. The model is trained with 100B tokens from the SlimPajama dataset tokenized with Llama2 tokenizer.
All-time downloads
656
Public
Parameters
1.4B
5.5 GB on disk
Likes
2
Public
Repo makeup
Click a slice to open those files.
.safetensors5.5 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- mit
- Model type
- gla
- Access
- Public
- Created
- Feb 12, 2024
- Updated
- Feb 12, 2024
- SHA
- 85e27cec
Try a prompt
Datasets
Model card
From the Hugging Face model README
This checkpoint of the 1.3B GLA model used in the paper Gated Linear Attention. The model is trained with 100B tokens from the SlimPajama dataset tokenized with Llama2 tokenizer.
See the model and loading script in this repo.
- Task
- Text Generation
- Library
- transformers
- Type
- gla
- License
- mit
- Languages
- en
- Created
- Feb 12, 2024
- Updated
- Feb 12, 2024