Typhon-Mixtral-v1-imatrix-v2.Q6_K-GGUF

• Contains: Q6_K GGUF (imatrix)
• Can fit in 16GB VRAM+32GB RAM, n_ctx 8k, 12 layers, blas 1024, reading speed generation

Mixtral 8x7B merge by <b>Sao10k</b>

(!!! ARCHIVE !!!)

<i>Calibrated imatrix data (V2 - 287kb) randomized bartowski, kalomeze groups, ERP/RP snippets, working gpt4 code, toxic qa, human messaging, randomized posts, story, novels</i>

---

Typhon - A Custom Experimental Mixtral Merge

An experimental Merge I tried for fun. Honestly did not expect it to work for Mixtral at all considering how its an MoE and the gates and all would be fucked by this custom merge.

From my testing it was able to handle SFW <--> NSFW scenarios fine, handle 1st and 3rd person roleplays fine, and seemed fairly smart.

It did pretty well for non NSFW tasks so that's a win.

Due to the nature of the merge, and Mixtral itself, it is sensitive to Prompts, does follow them well. Sampler settings are fine. i stuck with universal-light and was okay at up to 16k context during testing.

---

Recipe Below:

base_model: mistralai/Mixtral-8x7B-v0.1
models:
  - model: mistralai/Mixtral-8x7B-v0.1
    # no parameters necessary for base model
  - model: smelborp/MixtralOrochi8x7B
    parameters:
      weight: 0.30
      density: 0.47
  - model:  notstoic/Nous-Hermes-2-Mixtruct-v0.1-8x7B-DPO-DARE_TIES
    parameters:
      weight: 0.31
      density: 0.56
  - model: Sao10K/Solstice-Mixtral-v1
    parameters: 
      weight: 0.36
      density: 0.64
  - model: Sao10K/Frostwind-Mixtral-v1
    parameters:
      weight: 0.22
      density: 0.44
  - model: KoboldAI/Mixtral-8x7B-Holodeck-v1
    parameters:
      weight: 0.21
      density: 0.36
merge_method: dare_ties
dtype: bfloat16