Downloads · 30 days
16
15% of all-time downloads
aurelian-v0.5-70b-rope8-32K-2.4bpw_h6_exl2
grimulkan/aurelian-v0.5-70b-rope8-32K-2.4bpw_h6_exl2
aurelian-v0.5-70b-rope8-32K-2.4bpw_h6_exl2 is a text generation model from grimulkan. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as unknown.
This is a 2.4-bit EXL2 quantization of Aurelian v0.5 70B 32K, an interim checkpoint before v1.0. See that page for more details.
All-time downloads
104
Public
Repo size
21.3 GB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors21.3 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- unknown
- Model type
- llama
- Access
- Public
- Created
- Jan 14, 2024
- Updated
- Jan 14, 2024
- SHA
- 612ea26a
Try a prompt
Model card
From the Hugging Face model README
This is a 2.4-bit EXL2 quantization of Aurelian v0.5 70B 32K, an interim checkpoint before v1.0. See that page for more details.
This quantization fits in a single 24GB using Exllamav2 & 8-bit cache @ 10K context. It uses the newer experimental quantization method from turboderp.
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- unknown
- Created
- Jan 14, 2024
- Updated
- Jan 14, 2024