Downloads · 30 days
4
4% of all-time downloads
emozilla/scifi-fantasy-author-7b-8k_delta
scifi-fantasy-author-7b-8k_delta is a text generation model from emozilla. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
NOTE: This "delta model" cannot be used directly. Users have to apply it on top of the original LLaMA weights. See https://github.com/lm-sys/FastChatvicuna-weights for instructions. <br <br
Downloads · 30 days
4
4% of all-time downloads
All-time downloads
91
Public
Repo size
27 GB
Likes
3
Public
Click a slice to open those files.
.bin13.5 GB · 100%
From the Hugging Face model README
NOTE: This "delta model" cannot be used directly.
Users have to apply it on top of the original LLaMA weights.
See https://github.com/lm-sys/FastChat#vicuna-weights for instructions.
<br>
<br>
scifi-fantasy-author is a finetuned LLaMA-7B model to generate narrative fiction,
paricularly in the Science Fiction and Fantasy genres.
The following hyperparameters were used
| Batch Size | Epochs | Context length | Learning rate | Scheduler | Weight decay | Warmup ratio |
|---|---|---|---|---|---|---|
| 128 | 3 | 8192 | 2e-5 | Cosine | 0. | 0.03 |
The model reached a training loss of 2.008 and took approximately 8 hours on 8x A100 80 GB GPUs.
The specific training script can be found here.