Downloads · 30 days
10
4% of all-time downloads
Kooten/Noromaid-v0.4-Mixtral-Instruct-8x7b-Zloss-3.5bpw-exl2
Noromaid-v0.4-Mixtral-Instruct-8x7b-Zloss-3.5bpw-exl2 is a text generation model from Kooten. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as cc-by-nc-4.0.
Exllama quant of NeverSleep/Noromaid-v0.4-Mixtral-Instruct-8x7b-Zloss
Downloads · 30 days
10
4% of all-time downloads
All-time downloads
261
Public
Repo size
20.7 GB
Likes
5
Public
Click a slice to open those files.
.safetensors20.7 GB · 100%
From the Hugging Face model README
Exllama quant of NeverSleep/Noromaid-v0.4-Mixtral-Instruct-8x7b-Zloss
You will need 24gb of vram to run this model at about half context (16k, you can probably go a bit higher too)
<|im_start|>system
{sysprompt}<|im_end|>
<|im_start|>user
{input}<|im_end|>
<|im_start|>assistant
{output}<|im_end|>
Kooten on discord.