Downloads · 30 days
20
5% of all-time downloads
gpt2_774M_100B_FinewebEdu_hf
yuchenj/gpt2_774M_100B_FinewebEdu_hf
gpt2_774M_100B_FinewebEdu_hf is a text generation model from yuchenj. Use it when you need the model to write or continue text. It is set up for transformers.
This is a GPT-2 (774M) model trained in llm.c for 100B tokens with cosine LR on Fineweb-Edu.
All-time downloads
443
Public
Parameters
774M
1.5 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors1.5 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- gpt2
- Access
- Public
- Created
- Jul 26, 2024
- Updated
- Jul 26, 2024
- SHA
- 7b2c7826
Try a prompt
Datasets
Model card
From the Hugging Face model README
This is a GPT-2 (774M) model trained in llm.c for 100B tokens with cosine LR on Fineweb-Edu.
A lot more detailed info and observations are here: https://x.com/Yuchenj_UW/status/1814703583453192272
- Task
- Text Generation
- Library
- transformers
- Type
- gpt2
- Created
- Jul 26, 2024
- Updated
- Jul 26, 2024