Downloads · 30 days
80
5% of all-time downloads
NexesQuants/llama-3-70B-Instruct-abliterated-CQ-GGUF
llama-3-70B-Instruct-abliterated-CQ-GGUF is a machine learning model from NexesQuants. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Custom GGUF quant for : failspy/llama-3-70B-Instruct-abliterated-GGUF
Downloads · 30 days
80
5% of all-time downloads
All-time downloads
1.8K
Public
Repo size
168 GB
Likes
1
Public
Click a slice to open those files.
.gguf135 GB · 100%
From the Hugging Face model README
Custom GGUF quant for : failspy/llama-3-70B-Instruct-abliterated-GGUF
IQ4_SR is Optimal (8k ctx) for 36GB VRAM with an IGP displaying the OS.
IQ4_MR is optimal for the same config with MMQ and KV quants (8 bits)
Without an IGP, the IQ4_XSR is for you.