Downloads · 30 days
14
19% of all-time downloads
Goliath120B-exl2_2-2.64bpw
LavaPlanet/Goliath120B-exl2_2-2.64bpw
Goliath120B-exl2_2-2.64bpw is a text generation model from LavaPlanet. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as llama2.
Another EXL2 version of AlpinDale's https://huggingface.co/alpindale/goliath-120b this one being at 2.64BPW and using the new experimental quant method of exllamav2.
All-time downloads
73
Public
Repo size
39.5 GB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors39.5 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- llama2
- Model type
- llama
- Access
- Public
- Created
- Dec 15, 2023
- Updated
- Dec 15, 2023
- SHA
- e0061822
Try a prompt
Model card
From the Hugging Face model README
Another EXL2 version of AlpinDale's https://huggingface.co/alpindale/goliath-120b this one being at 2.64BPW and using the new experimental quant method of exllamav2.
Pippa llama2 Chat was used as the calibration dataset.
Can be run on two RTX 3090s w/ 24GB vram each.
Assuming Windows overhead, the following figures should be more or less close enough for estimation of your own use.
2.64BPW @ 4096 ctx
Empty Ctx
GPU Split:18/24
GPU1: 19.8/24
GPU2: 21.9/24
10~ tk/s
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- llama2
- Created
- Dec 15, 2023
- Updated
- Dec 15, 2023