Downloads · 30 days
4
7% of all-time downloads
LoneStriker/Llama-3-70B-Instruct-Storywriter-2.4bpw-h6-exl2
Llama-3-70B-Instruct-Storywriter-2.4bpw-h6-exl2 is a text generation model from LoneStriker. Use it when you need the model to write or continue text. It is set up for transformers.
Llama 3 70B Instruct, further finetuned on a dataset consisting of books in the fiction genre.
Downloads · 30 days
4
7% of all-time downloads
All-time downloads
54
Public
Repo size
23.5 GB
Likes
0
Public
Click a slice to open those files.
.safetensors23.5 GB · 100%
From the Hugging Face model README
Llama 3 70B Instruct, further finetuned on a dataset consisting of books in the fiction genre.
This was just an experiment, but it turned out well enough that I'm sharing it. The finetuning has caused a significant shift in the model's writing style, and seems to have made it more creative. There may be a slight decrease in overall intelligence.
Because this was trained on Instruct, you can use the normal Instruct chat formatting. It may also work well in raw completion mode.
Trained on 4 4090s using qlora-pipe. Dataset consists of about 800 books in the fiction genre, totaling 570 MB of raw text. Rank 64 QLoRA trained at 8192 sequence length.
I tried multiple times to train this on Llama 3 8B Instruct, using a variety of hyperparameters. It never worked well. The model took a huge hit to intelligence every time, to the point of being unusable. 70B fared much better. I don't know why, maybe 8B is just too small for this type of technique, and loses too much of the instruction-tuned smarts.