Downloads · 30 days
106
7% of all-time downloads
Dolly_GPT-J-6b
TehVenom/Dolly_GPT-J-6b
Dolly_GPT-J-6b is a text generation model from TehVenom. Use it when you need the model to write or continue text. It is set up for transformers.
This is a merge of the Dolly LoRA with the main GPT-J-6B model, allowing users to use Dolly without having to worry about PEFT dependencies.
All-time downloads
1.5K
Public
Repo size
24.4 GB
Likes
13
Public
Repo makeup
Click a slice to open those files.
.bin12.2 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- gptj
- Access
- Public
- Created
- Mar 27, 2023
- Updated
- Mar 27, 2023
- SHA
- e640043a
Try a prompt
Model card
From the Hugging Face model README
This is a merge of the Dolly LoRA with the main GPT-J-6B model, allowing users to use Dolly without having to worry about PEFT dependencies.
This hopes to be as similar as Alpaca, but without requirimg LLaMA access.
The performance is good but not as good as the orginal Alpaca trained from a base model of LLaMa
This is mostly due to the LLaMa 7B model being pretrained on 1T tokens and GPT-J-6B being trained on 300-400M tokens.
- LoRA originally trained by samwit, in: https://huggingface.co/samwit/dolly-lora
- The dataset is the cleaned version of the Alpaca dataset - https://github.com/gururise/AlpacaDataCleaned
- GPT-J-6b: https://huggingface.co/EleutherAI/gpt-j-6B
- here is a Colab https://colab.research.google.com/drive/1O1JjyGaC300BgSJoUbru6LuWAzRzEqCz?usp=sharing
- Task
- Text Generation
- Library
- transformers
- Type
- gptj
- Created
- Mar 27, 2023
- Updated
- Mar 27, 2023