Fish-8x7B-exl2-rpcal

Quantized using 200 samples of 8192 tokens from an RP-oriented PIPPA dataset.

Branches:

• main -- measurement.json
• 2.25b6h -- 2.25bpw, 6bit lm_head
• 3.5b6h -- 3.5bpw, 6bit lm_head
• 3.7b6h -- 3.7bpw, 6bit lm_head
• 5b6h -- 5bpw, 6bit lm_head
• 6b6h -- 6bpw, 6bit lm_head

Requires ExllamaV2 version 0.0.12 and up.

Original model link: Envoid/Fish-8x7B

This quant was broken. It is fixed now.

Original model README below.

---

Caution: This model may output adult content.

FISH - Fusion of Intuitive Semantic Heuristics

For this model I took the following LoRAs

SeanWu25/Mixtral_8x7b_Medicine

SeanWu25/Mixtral_8x7b_WuKurtz

and

wandb/Mixtral-8x7b-Remixtral

And merged them onto their base model.

I then did a simple linear merge between them, an experimental unreleased 8x7B model, and an unreleased model that was a intermediate step in creating Envoid/BondBurger-8x7B

The end results are a surprisingly good model for role palying style entertainment.

At first I was disappointed with the results but ended up settling on the following sampler parameters which really bring it to life.

It does eccentuate characters that have particularly aggressive personalities.

Because this is part of the whole SensualNousInstruct family of models it still suffers from the same tokenizer/special tokens weirdness.

This model has only been tested in Q8 GGUF form due to hardware limitations.

It responds well to [INST] do a thing [/INST] instruct style formatting (although uses the ChatML special tokens)