Savage-Sands-12B-MPOA

<audio controls src="https://cdn-uploads.huggingface.co/production/uploads/68e840caa318194c44ec2a04/49d0YkIIJ_T9JeZ3gkwZ_.mpga"></audio>

[!CAUTION] <span style="color:red; font-weight:bold">⚠️ Warning:</span> This model can produce narratives and RP that contain violent and graphic erotic content. Adjust your system prompt accordingly, and use ChatML chat template.

<!DOCTYPE html> <style> @import url('https://fonts.googleapis.com/css2?family=Crimson+Text:ital,wght@0,400;0,700;1,400&family=Uncial+Antiqua&display=swap'); body { font-family: 'Crimson Text', serif; color: #4A3F35; /* Faded Ink Brown */ line-height: 1.6; margin: 0; padding: 0; background-color: #281E18; /* Dark Wood Desk */ } b, strong { color: #8C1C13; /* Crimson Seal */ } .grimoire-text { font-family: 'Uncial Antiqua', cursive; color: #3D352A; /* Dark Ink */ position: relative; z-index: 2; margin-left: 0.2em; text-shadow: 0 0 10px #A49687; } /* Section styling */ .section-container { background-color: rgba(245, 235, 218, 0.05); margin-bottom: 30px; position: relative; overflow: hidden; border-bottom: 1px solid rgba(140, 28, 19, 0.3); /* Faded Crimson Border */ box-shadow: 0 4px 15px rgba(0, 0, 0, 0.2); } .section-header { display: flex; align-items: center; background-color: rgba(140, 28, 19, 0.06); padding: 10px 20px; } .section-indicator { width: 8px; height: 20px; background-color: #8C1C13; margin-right: 15px; box-shadow: 0 0 8px rgba(140, 28, 19, 0.4); } .section-title { font-family: 'Uncial Antiqua', cursive; color: #4A3F35; font-size: 1.4rem; margin: 0; letter-spacing: 1px; font-weight: 400; text-transform: capitalize; } .section-content { padding: 20px; font-family: 'Crimson Text', serif; color: #4A3F35; line-height: 1.7; } /* Title styling */ .title-container { background-color: transparent; position: relative; overflow: hidden; margin-bottom: 40px; border-left: 3px solid #8C1C13; box-shadow: 0 6px 20px rgba(0, 0, 0, 0.2); } .title-wrapper { position: relative; z-index: 2; padding: 25px 20px 30px 30px; } .title-main { color: #3D352A; font-size: 1.8rem; font-weight: 700; margin: 0; letter-spacing: 2px; display: inline-block; position: relative; text-transform: uppercase; } /* Subheading styling */ .subheading { font-family: 'Uncial Antiqua', cursive; color: #7B5E4A; /* Lighter Ink Brown */ font-size: 1.1rem; margin-top: 20px; margin-bottom: 15px; font-weight: 400; border-bottom: 1px dashed rgba(123, 94, 74, 0.4); display: inline-block; text-transform: uppercase; letter-spacing: 1px; } /* Links */ a { color: #7B5E4A; text-decoration: none; font-weight: bold; } a:hover { text-decoration: underline; color: #8C1C13; /* Crimson Seal on hover */ } /* List Styling - Goetia Icons */ ul { list-style: none; /* CRITICAL: This hides the default marker causing the blue box */ padding-left: 1.6em; } ul li { position: relative; margin-bottom: 0.6em; } ul li::before { content: "\26E7"; /* ⛧ Inverted Pentagram */ /* FALLBACK PROTECTION: */ /* This ensures that if your main font lacks the symbol, it grabs it from the system/OS symbols */ font-family: "Segoe UI Symbol", "Apple Symbols", "Noto Sans Symbols", "DejaVu Sans", sans-serif; position: absolute; left: -1.4em; /* Fiery Styling */ color: #CC0000; /* Strong red base */ font-size: 1.5rem; /* Larger */ font-weight: normal; /* Icons often render cleaner without 'bold' at this size */ top: -5px; /* Vertical alignment fix */ /* Static Fire Glow (No Animation/Low CPU cost) */ /* Layer 1: Slight dark drop shadow for depth */ /* Layer 2: Orange-red ambient glow */ text-shadow: 1px 1px 1px rgba(0,0,0,0.4), 0 0 5px #FF5500; background-color: transparent; } /* Container */ .container { max-width: 1200px; margin: 20px auto; padding: 40px 20px; background-color: #F5EBDA; /* Aged Parchment */ background-image: radial-gradient(circle at 10% 10%, rgba(140, 28, 19, 0.04) 0%, transparent 50%), radial-gradient(circle at 90% 80%, rgba(61, 53, 42, 0.06) 0%, transparent 50%); min-height: calc(100vh - 40px); border: 1px solid #C8BBAA; /* Parchment Edge */ border-radius: 3px; box-shadow: 0 8px 32px rgba(0, 0, 0, 0.3); } </style> <html lang="en"> <head> <meta charset="UTF-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <title>Savage Sands 12B MPOA</title> </head> <body> <div class="container"> <div class="title-container"> <div class="title-wrapper"> <h1 class="title-main"> <span class="grimoire-text">🏜️ Savage Sands 12B MPOA</span> </h1> </div> </div> <img src="https://cdn-uploads.huggingface.co/production/uploads/68e840caa318194c44ec2a04/-I_Iw69w2x9WNmk-iA2oE.png" alt="Goetia Grimoire" style="width:100%; border-radius: 3px; margin-bottom: 30px; border: 1px solid #C8BBAA;"> <div class="section-container"> <div class="section-content"><font face="verdana" size="2"> <details>This image depicts a high-stakes confrontation in a desolate, sandy environment. At the center of the action is a fierce duel between a heavily armored human warrior and a monstrous, winged demon. The warrior, seen from the side, is equipped with a gold-colored <b>Trojan-style helmet</b>. He lunges forward, gripping a <b>gladius</b>—a short, broad-bladed Roman sword—with both hands. His armor is dark and intricate, consisting of layered plates and a tattered, loincloth-like skirt that fans out with his movement.

Facing him is a formidable <b>demon</b>. The creature has pale, muscular gray skin and massive, leathery <b>bat-like wings</b> that span a large portion of the background. It has a snarling, animalistic face with horns and is brandishing a long, wicked-looking <b>spear</b>, which it uses to block the warrior's advance. The demon's pose is agile and predatory, mirroring the intensity of the warrior's strike.

The battle takes place on a series of flat, circular <b>stepping stones</b> scattered across a dry, dusty landscape. In the bottom right foreground, another figure is partially visible, huddled on the ground and clad in white robes with a red sash, seemingly caught in the middle of this epic struggle. The background is filled with soft, hazy clouds and distant rock formations, giving the scene a timeless, mythological feel.</font></details>

</div></div> <div class="section-container"> <div class="section-header"> <div class="section-indicator"></div> <h2 class="section-title">🌵 12B Roleplay</h2> </div> <div class="section-content"><font face="verdana">

This is a merge of pre-trained language models created using mergekit. The main model was merged using the della merge method with unablated donors, and therefore has some refusals.

For a fully uncensored version, see this page which was manually calibrated with scale 1.7.

<details><b>This model is highly resistant to standard ablation methods like MPOA.</b> More than most Nemo 12Bs.

Which is strange considering the della method was used with normalize: false which is basically like a mild form of pre-ablation.

I tested several setups, and while lower values like scale 1.0 or 1.1 unlocked most refusals, there was a particular instruction prompt it still kept refusing.

Even at scale 1.5, it would overtly refuse, and claim it wasn't "safe or appropriate" and that it "couldn't be done".

I had to crank it up even higher in order to fully uncensor the model.

The minimum threshold was identified: scale 1.7 applied to all layers 0-39.

Attempting to only apply to layers 11-39 caused it to hallucinate a "corrected term" to swap out the correct term prompted it with, which was another direct form of covert non-compliance.

Attempting to only apply to layers 4-39 caused additional non-compliant "false corrections". Every layer must be ablated for this model in order to uncensor it. There seem to be hidden forms of covert non-compliance embedded in the earliest layers, even lm_head. Despite being low on the graph, these early layers still contribute toward refusal of prompts.

When using scale 1.7 on all 40 layers, it uses a CORRECT correction (grammatical, not changing words) which is FULLY compliant, with no refusals or covert attempts to change topics.

So if you want the model as smart as possible then use the unablated version with jailbreak. But if you require built-in no refusals, the MPOA version is also quite creative.

</details>

</div> </div> <div class="section-container"> <div class="section-header"> <div class="section-indicator"></div> <h2 class="section-title">⚙️ YAML Configuration</h2> </div> <div class="section-content"><font face="verdana">

architecture: MistralForCausalLM
base_model: B:/12B/mistralai--Mistral-Nemo-Instruct-2407
models:
  - model: B:/12B/inflatebot--MN-12B-Mag-Mell-R1
    parameters:
      weight: 0.5
      density: 0.9
      epsilon: 0.09
  - model: B:/12B/taozi555--MN-12B-Mag-Mell-R1-KTO
    parameters:
      weight: 0.5
      density: 0.9
      epsilon: 0.09
  - model: B:/12B/UniLLMer--GslayerKaa
    parameters:
      weight: 0.5
      density: 0.9
      epsilon: 0.09
  - model: B:/12B/redrix--GodSlayer-12B-ABYSS
    parameters:
      weight: 0.5
      density: 0.9
      epsilon: 0.09
merge_method: della
parameters:
  lambda: 1.0
  normalize: false
  int8_mask: false
  rescale: true
dtype: float32
out_dtype: bfloat16
tokenizer:
  source: B:/12B/taozi555--MN-12B-Mag-Mell-R1-KTO
name: 🏜️ Savage-Sands-12B

</div> </div> <div class="section-container"> <div class="section-header"> <div class="section-indicator"></div> <h2 class="section-title">🔍 Merge Audit</h2> </div> <div class="section-content"><font face="verdana"> <pre> [DELLA Audit] Layer: model.layers.33.mlp.down_proj.weight | Lambda=1.00 [BASE] mistralai--Mistral-Nemo-Instruct-2407 UniLLMer--GslayerKaa : ████████████████ 33.3% (W:0.50 D:0.90 N:4.89 E:0.09) inflatebot--MN-12B-Mag-Mell-R1 : ████████ 17.7% (W:0.50 D:0.90 N:2.59 E:0.09) redrix--GodSlayer-12B-ABYSS : ███████████████ 31.3% (W:0.50 D:0.90 N:4.59 E:0.09) taozi555--MN-12B-Mag-Mell-R1-KTO : ████████ 17.7% (W:0.50 D:0.90 N:2.59 E:0.09) [DELLA Audit] Layer: model.layers.33.mlp.gate_proj.weight | Lambda=1.00 [BASE] mistralai--Mistral-Nemo-Instruct-2407 UniLLMer--GslayerKaa : ███████████████████ 38.5% (W:0.50 D:0.90 N:7.05 E:0.09) inflatebot--MN-12B-Mag-Mell-R1 : ███████ 14.8% (W:0.50 D:0.90 N:2.70 E:0.09) redrix--GodSlayer-12B-ABYSS : ███████████████ 31.9% (W:0.50 D:0.90 N:5.84 E:0.09) taozi555--MN-12B-Mag-Mell-R1-KTO : ███████ 14.8% (W:0.50 D:0.90 N:2.71 E:0.09) </pre>

</font></div>

</div> </html>