Downloads · 30 days
19
4% of all-time downloads
Llama-3.1-Base_NSFW-pretrained_e-0.5
athirdpath/Llama-3.1-Base_NSFW-pretrained_e-0.5
Llama-3.1-Base_NSFW-pretrained_e-0.5 is a text generation model from athirdpath. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as llama3.1.
Llama 3.1 Base, continually pretrained with 0.5 Epochs (2100 steps @ total batch 64) of the same 1.5gb private dataset that underpins Iambe
All-time downloads
518
Public
Parameters
8B
16.1 GB on disk
Likes
2
Public
Repo makeup
Click a slice to open those files.
.safetensors16.1 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- llama3.1
- Model type
- llama
- Access
- Public
- Created
- Jul 27, 2024
- Updated
- Jul 30, 2024
- SHA
- 64147701
Try a prompt
Model card
From the Hugging Face model README
Llama 3.1 Base, continually pretrained with 0.5 Epochs (2100 steps @ total batch 64) of the same 1.5gb private dataset that underpins Iambe
Mostly a proof of concept, but outputs are better than expected. It'd likely be quite good with some instruction tuning.
Why do this? I have a niche use case where I cannot increase compute over 8b, and L3/3.1 are the only models in this size category that meet my needs for logic. However, both versions of L3/3.1 have the damn repetition/token overconfidence problem, and this is meant to disrupt that certainty without disrupting the model's ability to function.
By the way, I think it's the lm_head that is causing the looping, but it might be the embeddings being too separated. I'm not going to pay two more times to test them separately, however :p
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- llama3.1
- Created
- Jul 27, 2024
- Updated
- Jul 30, 2024