Downloads · 30 days
8
15% of all-time downloads
Kooten/Noromaid-v0.1-mixtral-8x7b-Instruct-v3-4bpw-exl2
Noromaid-v0.1-mixtral-8x7b-Instruct-v3-4bpw-exl2 is a text generation model from Kooten. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as cc-by-nc-4.0.
Exllama quant of NeverSleep/Noromaid-v0.1-mixtral-8x7b-Instruct-v3
Downloads · 30 days
8
15% of all-time downloads
All-time downloads
54
Public
Repo size
23.6 GB
Likes
0
Public
Click a slice to open those files.
.safetensors23.6 GB · 100%
From the Hugging Face model README
Exllama quant of NeverSleep/Noromaid-v0.1-mixtral-8x7b-Instruct-v3
You probably want to use the 3.5bpw version, it lands very close to 24gb of vram with 8bit cache enabled. GGUF is probably preferable if you have less than 24gb.
Other BPW's 3.0bpw, 3.5bpw, 4.0bpw
### Instruction:
{system prompt}
### Input:
{input}
### Response:
{reply}
Kooten on discord.