Downloads · 30 days
18
5% of all-time downloads
llama33b-s2a4
chargoddard/llama33b-s2a4
llama33b-s2a4 is a text generation model from chargoddard. Use it when you need the model to write or continue text. It is set up for transformers.
LLaMA 33b finetuned on wikitextdocumentlevel with combined linear and NTK-aware ROPE scaling (alpha=4, scale=2.) This model will be coherent up to at least 8k context length, but might work beyond that. This is a merg…
All-time downloads
331
Public
Repo size
130 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin65.1 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- llama
- Access
- Public
- Created
- Jul 18, 2023
- Updated
- Jul 18, 2023
- SHA
- ebf85fc4
Try a prompt
Model card
From the Hugging Face model README
LLaMA 33b finetuned on wikitext_document_level with combined linear and NTK-aware ROPE scaling (alpha=4, scale=2.)
This model will be coherent up to at least 8k context length, but might work beyond that.
This is a merged version of llama33b-s2a4-qlora.
Note that this is not an instruct model - this is base LLaMA with an extended sequence length.
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- Created
- Jul 18, 2023
- Updated
- Jul 18, 2023