Downloads · 30 days
18
4% of all-time downloads
32M-32GT-SlimPajama
crumb/32M-32GT-SlimPajama
32M-32GT-SlimPajama is a text generation model from crumb. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
checkpoint @ 31,404,851,200 tokens trained, settings used:
All-time downloads
403
Public
Repo size
282 MB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.bin282 MB · 99%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- gpt2a
- Access
- Public
- Created
- Dec 6, 2023
- Updated
- Dec 6, 2023
- SHA
- ce7a2c78
Try a prompt
Datasets
Model card
From the Hugging Face model README
checkpoint @ 31,404,851,200 tokens trained, settings used:
batch_size: 512
context_length: 1024
learning_rate: 2e-4
schedule: cosine with 10% warmup from 0 to 2e-4, cooldown to 0
requirements:
pytorch
transformers
einops
- Task
- Text Generation
- Library
- transformers
- Type
- gpt2a
- License
- apache-2.0
- Languages
- en
- Created
- Dec 6, 2023
- Updated
- Dec 6, 2023