Downloads · 30 days
97
1% of all-time downloads
gpt-j-6B
flyhero/gpt-j-6B
gpt-j-6B is a text generation model from flyhero. Use it when you need the model to write or continue text. It is set up for transformers.
GPT-J 6B is a transformer model designed using EleutherAI's replication of the GPT-3 architecture. GPT-J refers to the class of models, while 6B represents the number of parameters of this particular pre-trained model.
All-time downloads
11.5K
Public
Repo size
25.1 GB
Likes
13
Public
Repo makeup
Click a slice to open those files.
.bin12.6 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- gpt_neo
- Access
- Public
- Created
- Mar 2, 2022
- Updated
- Aug 19, 2021
- SHA
- c7aaf3b7
Try a prompt
Model card
From the Hugging Face model README
Model Description
GPT-J 6B is a transformer model designed using EleutherAI's replication of the GPT-3 architecture. GPT-J refers to the class of models, while 6B represents the number of parameters of this particular pre-trained model.
The original GPT-J-6B model is trained with TPUs, which is not easy to use for normal users. Thus, through a converting script, we convert the TPU version GPT-J-6B into GPU version, which could be load and fine-tuned with GPUs.
As we have tried, the model can be loaded with 1 GPU with 16G memory to do inference. For fine-tune, we used 8 * 32G GPUs with DeepSpeed library to distribute the model, data and gradients, in order to allocate the huge amount of model parameters.
- Task
- Text Generation
- Library
- transformers
- Type
- gpt_neo
- Created
- Mar 2, 2022
- Updated
- Aug 19, 2021