Downloads · 30 days
20
6% of all-time downloads
brucethemoose/Yi-34B-200K-elx2-39bpw
Yi-34B-200K-elx2-39bpw is a text generation model from brucethemoose. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as other.
See: https://huggingface.co/01-ai/Yi-34B-200K
Downloads · 30 days
20
6% of all-time downloads
All-time downloads
319
Public
Repo size
17.6 GB
Likes
1
Public
Click a slice to open those files.
.safetensors17.6 GB · 100%
From the Hugging Face model README
See: https://huggingface.co/01-ai/Yi-34B-200K
Yi-30B-200K quantized to 3.9bpw, which should allow for ~50K context on 24GB GPUs. Ask if you need another size.
Quantized with 8K rows on a mix of wikitext, prompt formatting, and my own RP stories.
Use with --enable-remote-code in text-gen-ui. Load with Exllamav2_HF, use 8-bit cache, and disable the fast_tokenizer option. The TFS preset seems to work well with Yi.