Downloads · 30 days
14
19% of all-time downloads
grimulkan/aurelian-v0.5-70b-rope8-32K-4.65bpw_h6_exl2
aurelian-v0.5-70b-rope8-32K-4.65bpw_h6_exl2 is a text generation model from grimulkan. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as unknown.
This is a 4-bit EXL2 quantization of Aurelian v0.5 70B 32K, an interim checkpoint before v1.0. See that page for more details.
Downloads · 30 days
14
19% of all-time downloads
All-time downloads
74
Public
Repo size
40.5 GB
Likes
1
Public
Click a slice to open those files.
.safetensors40.5 GB · 100%
From the Hugging Face model README
This is a 4-bit EXL2 quantization of Aurelian v0.5 70B 32K, an interim checkpoint before v1.0. See that page for more details.
This quantization fits in 2x24GB (19/24) using Exllamav2 @ 16K context.