Downloads · 30 days
21
4% of all-time downloads
llama3-wordcel
jspr/llama3-wordcel
llama3-wordcel is a text generation model from jspr. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
Wordcel is a Llama3 fine-tune intended to be used as a mid-training checkpoint for more specific RP/storywriting/creative applications.
All-time downloads
502
Public
Parameters
8B
16.1 GB on disk
Likes
7
Public
Repo makeup
Click a slice to open those files.
.safetensors16.1 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- llama
- Access
- Public
- Created
- May 2, 2024
- Updated
- May 2, 2024
- SHA
- d74f9d13
Try a prompt
Base models
Model card
From the Hugging Face model README
Llama3 8B Wordcel
Wordcel is a Llama3 fine-tune intended to be used as a mid-training checkpoint for more specific RP/storywriting/creative applications.
It has been trained from Llama3 8B Base on a composite dataset of ~100M tokens that highlights reasoning, (uncensored) stories, classic literature, and assorted interpersonal intelligence tasks.
Components of the composite dataset include OpenHermes-2.5, and Grimulkan's Theory of Mind and Physical Reasoning datasets.
It is trained at a context length of 32k tokens, using linear RoPE scaling with a factor of 4.0. Derivative models should be capable of generalizing to 32k tokens as a result.
If you train a model using this checkpoint, please give clear attribution! The Llama 3 base license likely applies.
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- apache-2.0
- Languages
- en
- Created
- May 2, 2024
- Updated
- May 2, 2024