Downloads · 30 days
5
2% of all-time downloads
lmganon123/Euryale-L2-70B-2.1BPW-exllama2
Euryale-L2-70B-2.1BPW-exllama2 is a text generation model from lmganon123. Use it when you need the model to write or continue text. It is set up for transformers.
Quantized : https://huggingface.co/Sao10K/Euryale-L2-70B
Downloads · 30 days
5
2% of all-time downloads
All-time downloads
256
Public
Parameters
5B
19.4 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors19.4 GB · 100%
How the weights are stored.
I324.7B · 95%
From the Hugging Face model README
Quantized : https://huggingface.co/Sao10K/Euryale-L2-70B
With : https://github.com/turboderp/exllamav2
Fits into 24GB of ram with 4096 context. Unfortunately it seems to be dumbed down a bit too much by compression. Files also include measurement.json that can be used to speed up quantization process for other BPW size.
Measurement done with default parameters and https://huggingface.co/datasets/wikitext/tree/refs%2Fconvert%2Fparquet/wikitext-103-raw-v1/test