RPBizkit-v8-12B

RicardoEstep/RPBizkit-v8-12B

"Experimental Mistral Nemo Mix" using... "Model Stock + SLERP merges with Mergekit".

This is a mix of some known "RP Uncensored" models. I'm not responsible of the use of this.

I did it all for the nookie, so you can take that cookie, and stick it up your...

¡This merge is based in 3 parts!

After noticing the f*****g "chaotic" last version, that works at worst... I decided to go "even simplier". I had to reduce the overall models as some are "Mixes from a Mix", i tried my best using only "FineTunes".

Part One: "RP Core".

Models Merged:

• natong19/Mistral-Nemo-Instruct-2407-abliterated # Base!
• TheDrummer/UnslopNemo-12B-v4.1
• ArliAI/Mistral-Nemo-12B-ArliAI-RPMax-v1.2
• allura-org/MN-12b-RP-Ink
• nbeerbower/mistral-nemo-gutenberg-12B-v4

Configuration Used:

merge_method: model_stock
base_model: natong19/Mistral-Nemo-Instruct-2407-abliterated
dtype: bfloat16
tokenizer:
  source: base
models:
  - model: TheDrummer/UnslopNemo-12B-v4.1
  - model: nbeerbower/mistral-nemo-gutenberg-12B-v4
  - model: ArliAI/Mistral-Nemo-12B-ArliAI-RPMax-v1.2
  - model: allura-org/MN-12b-RP-Ink

Part Two: "Substance Core".

Models Merged:

• natong19/Mistral-Nemo-Instruct-2407-abliterated # Base!
• SicariusSicariiStuff/Impish_Bloodmoon_12B
• ChaoticNeutrals/Nera_Noctis-12B
• allura-org/Bigger-Body-12b
• ReadyArt/Forgotten-Safeword-12B-v4.0

Configuration Used:

merge_method: model_stock
base_model: natong19/Mistral-Nemo-Instruct-2407-abliterated
dtype: bfloat16
tokenizer:
  source: base
models:
  - model: ReadyArt/Forgotten-Safeword-12B-v4.0
  - model: SicariusSicariiStuff/Impish_Bloodmoon_12B
  - model: ChaoticNeutrals/Nera_Noctis-12B
  - model: allura-org/Bigger-Body-12b

Part Three: "The Final Mix".

Models Merged:

• .\mergekit_config1 # RP Core
• .\mergekit_config2 # Substance

Configuration Used:

merge_method: slerp
base_model: .\mergekit_config1  # RP Core
dtype: bfloat16
slices:
  - sources:
      - model: .\mergekit_config1  # RP Core
        layer_range: [0, 40]
      - model: .\mergekit_config2  # Substance
        layer_range: [0, 40]
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.27, 0]  # Priority on RP Core logic.
    - filter: mlp
      value: [1, 0.5, 0.7, 0.73, 1]  # Priority on Substance Core Vocab.
    - value: 0.5  # "50/50 Mix" for rest.

Final Notes;

The Tokenizer & Embedding Sizes (131072) are clean + It's "supposed to" support now a full "128K Context Size".

The recommended chat template to use is "Alpaca (with "RAW" inputs)". The configuration files are already "Tweaked", to don't use any "Chat Template". But, now, it's "supposed to" full support too, the "optional_chat_template.jinja.7z (Mistral V3)" if is decompressed.

• Quantized Model, Here RicardoEstep/RPBizkit-v8-12B-GGUF.