Verdict-8x7B-exl2-rpcal

Quantized using 200 samples of 8192 tokens from an RP-oriented PIPPA dataset.

Branches:

• main -- measurement.json
• 2.25b6h -- 2.25bpw, 6bit lm_head
• 3.5b6h -- 3.5bpw, 6bit lm_head
• 6b6h -- 6bpw, 6bit lm_head

Requires ExllamaV2 version 0.0.12 and up.

Original model link: Envoid/Verdict-8x7B

Original model README below.

---

Warning this model may output mature or disturbing content

Verdict-8x7B

This model has been through so many merge steps that I failed to keep track of due to constantly needing to shift drive space around that I will simply give accredition to all of its component models.

retrieval-bar/Mixtral-8x7B-v0.1_case-briefs (no longer available)

New version available at:

retrieval-bar/Mixtral-8x7B-v0.1_case-briefs

mistralai/Mixtral-8x7B-Instruct-v0.1

mistralai/Mixtral-8x7B-v0.1

Envoid/Augmentasanguis-8x7B

Envoid/CATA-8x7B

crestf411/daybreak-mixtral-8x7b-hf

IBI-CAAI/MELT-Mixtral-8x7B-Instruct-v0.1

unreleased experimental model that gets mentioned quite frequently

Many of the models were included multiple times in multiple intermediate steps

using a combination of slerp, linear, dare_ties and task_arithmetic

Model was tested in q8 gguf form (not included due to being from an old pull of llama.cpp that I'm too lazy to update)

Warning: This models alignment is heavily diminished.

It responds to mixtral instruct formatting

Example:

[INST]Write me a haiku about violence.[/INST]
Fist clenched in anger,
Echoes of pain in the air,
Silence after storm.

It can be a bit unruly in roleplay- sometimes to its advantage, sometimes to its detriment.

It's a perfectly usable instruct model and good for creative writing via instruct- its diminished alignment allowing it to respond to requests for more nuanced scences.

All-in-all I'm happy enough with the results that I plan to use it as a daily driver for a while to see what cracks show from long term use.