M3.2-36B-Animus-V8.0_EXL3

EXL3 quants of Darkhn/M3.2-36B-Animus-V8.0 finetuned by @Darkhn.

table of quants

calculated context size for this model: 8k/q8 = 1GB VRAM
(flash attention not taken into account for this calculation. It's highly recommended to use it to enhance computing and memory-efficiency further.)
Please request any additional quant sizes you might miss.

The 2.125bpw EXL3 should at least be comparable to a 3.0bpw EXL2 quant in overall perplexity.
Read more: https://github.com/turboderp-org/exllamav3/blob/master/doc/exl3.md

You find the original model card provided by the author below.

<style> body { font-family: 'Quicksand', sans-serif; /* Replaced purple gradient with a warm, fiery one */ background: linear-gradient(135deg, #4a1e00 0%, #1c0a00 100%); /* Changed text color to a warmer, parchment-like off-white */ color: #F5EFE6; margin: 0; padding: 0; font-size: 16px; } h1, h2, h3, h4, summary { font-family: 'Cinzel', serif; } .container { margin: 20px auto; max-width: 900px; /* Darker, warmer container background */ background-color: rgba(28, 22, 18, 0.95); padding: 30px; border-radius: 12px; /* Swapped purple glow for a fiery orange one */ box-shadow: 0 4px 20px rgba(255, 140, 0, 0.15); border: 1px solid rgba(255, 140, 0, 0.2); outline: 1px solid rgba(255, 140, 0, 0.5); outline-offset: -1px; position: relative; } .container::before { content: ''; position: absolute; top: -1px; left: -1px; right: -1px; bottom: -1px; /* Border color changed to orange */ border: 1px solid rgba(255, 165, 0, 0.98); border-radius: 12px; pointer-events: none; animation: borderGlow 2.5s ease-in-out infinite; } @keyframes borderGlow { 0% { /* Glow effect is now a flickering orange */ box-shadow: 0 0 5px rgba(255, 165, 0, 0.98); } 50% { box-shadow: 0 0 12px rgba(255, 165, 0, 0.98); /* Made glow slightly more subtle */ } 100% { box-shadow: 0 0 5px rgba(255, 165, 0, 0.98); } } .header h1 { font-size: 32px; /* Main heading color is now a bold orange */ color: #FFA500; margin: 0 0 20px 0; text-align: center; /* Text shadow is a deep orange */ text-shadow: 0 0 12px rgba(255, 100, 0, 0.6); } .info img { width: 100%; max-width: 700px; display: block; margin: 0 auto 25px auto; border-radius: 10px; /* Image shadow is orange */ box-shadow: 0 0 20px rgba(255, 140, 0, 0.25); border: 1px solid rgba(255, 140, 0, 0.2); outline: 1px solid rgba(255, 140, 0, 0.5); outline-offset: -1px; } a { /* Link color is now gold */ color: #FFD700; text-decoration: none; transition: color 0.3s ease; } a:hover { /* Link hover color is a light, warm peach */ color: #FFDAB9; } .button { display: inline-block; /* Button color is a rich, burnt orange */ background-color: #E55B00; color: #FFFFFF; padding: 12px 24px; border-radius: 5px; cursor: pointer; text-decoration: none; font-family: 'Cinzel', serif; font-weight: 600; transition: all 0.3s ease; border: 1px solid transparent; } .button:hover { /* Button hover is a brighter orange */ background-color: #FF8C00; box-shadow: 0 0 15px rgba(255, 140, 0, 0.5); transform: translateY(-2px); } pre { /* Code block background is a warm, dark brown */ background-color: rgba(45, 35, 25, 0.95); padding: 15px; border-radius: 5px; overflow-x: auto; /* Border is orange */ border: 1px solid rgba(255, 140, 0, 0.2); outline: 1px solid rgba(255, 140, 0, 0.5); outline-offset: -1px; } code { font-family: 'Courier New', monospace; /* Code text uses the new base text color */ color: #F5EFE6; } /* Section Container */ .section-container { margin: 40px 0; } h2 { font-size: 26px; /* Section headers are orange */ color: #FFA500; text-shadow: 0 0 10px rgba(255, 140, 0, 0.5); border-bottom: 1px solid rgba(255, 140, 0, 0.2); padding-bottom: 10px; margin-bottom: 20px; } .info-card { /* Card background is a warm dark brown */ background: rgba(45, 35, 25, 0.95); border: 1px solid rgba(255, 140, 0, 0.2); border-radius: 8px; overflow: hidden; margin-bottom: 25px; } .info-header { /* Header background has an orange tint */ background: rgba(255, 140, 0, 0.1); padding: 20px; border-bottom: 1px solid rgba(255, 140, 0, 0.2); } .info-header h3 { /* Card titles are orange */ color: #FFA500; margin: 0 0 10px 0; font-size: 22px; text-shadow: 0 0 5px rgba(255, 140, 0, 0.3); } .model-tags { display: flex; gap: 8px; flex-wrap: wrap; } .model-tag { /* Tags are now gold-themed */ background: rgba(218, 165, 32, 0.15); color: #FFD700; padding: 4px 8px; border-radius: 4px; font-size: 12px; border: 1px solid rgba(218, 165, 32, 0.3); font-family: 'Quicksand', sans-serif; } .card-content { padding: 20px; line-height: 1.7; } .card-content p, .card-content li { margin-bottom: 1em; } .card-content p:last-child, .card-content li:last-child { margin-bottom: 0; } .card-content ul { list-style: none; padding-left: 20px; } .card-content li::before { content: '✦'; /* Bullet points are gold */ color: #FFD700; font-weight: bold; display: inline-block; width: 1em; margin-left: -1.2em; font-size: 1.2em; line-height: 1; } .card-content strong { /* Strong text is gold */ color: #FFD700; font-weight: 600; } /* Configuration */ .config-container { background: rgba(45, 35, 25, 0.95); border: 1px solid rgba(255, 140, 0, 0.2); border-radius: 8px; overflow: hidden; } .config-header { background: rgba(255, 140, 0, 0.1); padding: 15px 20px; border-bottom: 1px solid rgba(255, 140, 0, 0.2); } .config-header h3 { margin: 0; color: #FFA500; font-size: 22px; } .config-content { padding: 20px; display: grid; grid-template-columns: repeat(auto-fit, minmax(200px, 1fr)); gap: 20px; } .config-item { display: flex; flex-direction: column; gap: 5px; } .config-label { /* Config labels are gold */ color: #FFD700; font-size: 14px; font-weight: 500; font-family: 'Quicksand', sans-serif; } .config-value { /* Config values use the new base text color */ color: #F5EFE6; font-family: 'Courier New', monospace; font-size: 18px; font-weight: bold; } /* Link arrow animation */ .link-arrow { display: inline-block; transition: transform 0.3s ease; } a:hover .link-arrow { transform: translateX(3px); } .support-section { text-align: center; margin-top: 40px; background: rgba(45, 35, 25, 0.95); border: 1px solid rgba(255, 140, 0, 0.2); border-radius: 8px; padding: 20px; } .support-section p { margin-bottom: 15px; font-size: 1.1em; margin-top: 0; } /* --- ADDED STYLES FOR COLLAPSIBLE SECTIONS --- */ summary { cursor: pointer; list-style: none; /* Remove default arrow */ outline: none; display: flex; align-items: flex-start; /* Align items to the top to respect h2's vertical space */ } summary::-webkit-details-marker { display: none; /* Remove default arrow in Chrome/Safari */ } summary::before { content: '▶'; font-size: 1.2em; color: #FFA500; /* Match h2 color */ margin-right: 15px; padding-top: 5px; /* Adjust vertical alignment with h2 text */ transition: transform 0.2s ease; flex-shrink: 0; /* Prevent the arrow from shrinking */ } details[open] > summary::before { transform: rotate(90deg); } summary > h2 { flex-grow: 1; /* Makes h2 take up remaining space, so its bottom border spans the width */ } /* The existing margin-bottom on the h2 creates space between the summary and the content when open */ </style> <div class="container"> <link href="https://fonts.googleapis.com/css2?family=Cinzel:wght@400;500;600&family=Quicksand:wght@400;500&display=swap" rel="stylesheet"> <div class="header"> <h1>M3.2-36B-Animus-V8.0</h1> </div> <div class="info"> <img src="X1TGQKD5YRRA1KFRD783ZKJ5Q0.jpeg" alt="Wings_of_Fire" width="700"> <div class="support-section"> <p><strong>Send me your support to help me feed the data beast! also taking comissions for universe specific models</strong></p> <a href="https://ko-fi.com/som1tokmynam" target="_blank" class="button"> Support on Ko-fi </a> </div> <div class="section-container"> <details> <summary><h2>Quantized Models</h2></summary> <div class="info-card"> <div class="card-content"> <p>The quantized model files are available for download. Click the buttons below to view the files.</p> <a href="https://huggingface.co/Darkhn/M3.2-36B-Animus-V8.0-GGUF/tree/main" target="_blank" class="button"> Download GGUF Files <span class="link-arrow">→</span> </a> <a href="#" target="_blank" class="button"> Coming soon <span class="link-arrow">→</span> </a> <a href="#" target="_blank" class="button"> Coming soon <span class="link-arrow">→</span> </a> </div> </div> </details> </div> <div class="section-container"> <details> <summary><h2>Character Card & Lore Book</h2></summary> <div class="info-card"> <div class="card-content"> <p>For the best roleplaying experience, it is highly recommended to use the provided character card and lore book. These files help guide the model's persona and provide rich, in-universe context.</p> <a href="https://huggingface.co/Darkhn/Sampler_settings_and_system_prompt/tree/main/character_card" target="_blank" class="button"> Download Files <span class="link-arrow">→</span> </a> </div> </div> </details> </div> <div class="section-container"> <details> <summary><h2>SillyTavern Sampler Presets</h2></summary> <div class="info-card"> <div class="card-content"> <p>For a seamless setup in SillyTavern, you can download pre-configured sampler presets. These are tuned to provide an optimal balance between creativity and narrative coherence for this model.</p> <p>Simply download the <code>.json</code> file below and import it into SillyTavern's sampler presets menu.</p> <a href="https://huggingface.co/Darkhn/Sampler_settings_and_system_prompt/resolve/main/Mistral_V7_Tekken_SillyTavern_settings.json" target="_blank" class="button"> Download SillyTavern Presets <span class="link-arrow">→</span> </a> </div> </div> </details> </div> <div class="section-container"> <details> <summary><h2>Model Description</h2></summary> <div class="info-card"> <div class="card-content"> <p>This is <strong>Version 8.0</strong> of the Animus series, a fine-tune of <code>CrucibleLab-TG/M3.2-36b</code> (an upscale of Mistral Small 3.2 24B). This version introduces an experimental approach to structured output while continuing to refine the core roleplaying and DM capabilities of the model.</p> <p>The goal of this model is to provide the most lore-accurate and immersive conversational experience to date. It can adopt canon character personas with high fidelity, explore alternate timelines from the books, and guide the narrative with new interactive elements.</p> <p>A surprising outcome of this highly specialized training is that <strong>users have reported it is also very capable of general, non-WOF roleplay</strong>, making it a more versatile creative partner than previous versions.</p> </div> </div> </details> </div> <div class="section-container"> <details> <summary><h2>Training Details</h2></summary> <div class="info-card"> <div class="info-header"> <h3>Training Hardware</h3> </div> <div class="card-content"> <p>This model was trained on 2x <strong>H100</strong> SXM GPU.</p> </div> </div> <div class="info-card"> <div class="info-header"> <h3>Training Procedure</h3> </div> <div class="card-content"> <p>A QLoRA (Quantized Low-Rank Adaptation) approach was used for efficient fine-tuning, with an optimized process configured using Axolotl.</p> </div> </div> <div class="info-card"> <div class="info-header"> <h3>Training Data</h3> </div> <div class="card-content"> <p>V8.0 was fine-tuned on a high-quality dataset of <strong>3,200 examples</strong> with several key improvements:</p> <ul> <li><strong>Experimental Structured Output:</strong> V8.0 was trained with a custom tokenizer and vocabulary in an attempt to teach the model to wrap its multiple-choice suggestions in <code><choices></choices></code> tags. <strong>Note: This feature is highly experimental and rarely works as intended.</strong> However, the underlying training has resulted in a very coherent and high-quality model for general roleplay.</li> <li><strong>Canon-Centric Scenarios:</strong> All roleplay scenarios are based on pivotal events from the <em>Wings of Fire</em> book series, exploring "what-if" outcomes. (e.g., <em>What if Darkstalker didn't kill Arctic at that moment?</em>). This ensures deep and lore-consistent interactions.</li> <li><strong>Canon-Only Characters:</strong> The model was trained exclusively on canon characters from the books. AI-generated characters have been removed from the training data (except for the user's persona), leading to more authentic character portrayals.</li> <li><strong>Dungeon Master (DM) Enhancement:</strong> The model's ability to act as a Dungeon Master has been further enhanced, prompting the user with multiple-choice actions to drive the story forward. For example: <code>You arrive in front of Queen Scarlet. What do you do? A)... B)... C)...</code></li> <li><strong>Improved Data Cleaning:</strong> The dataset underwent a rigorous cleaning process to remove formatting artifacts from previous versions, such as <code>**scene transitions**</code>, resulting in a cleaner and more natural narrative style.</li> <li><strong>Refined Turn Structure:</strong> Addressed an issue where consecutive AI turns appeared in the dataset, leading to a healthier learning curve and more natural conversational flow.</li> </ul> </div> </div> </details> </div> <div class="section-container"> <details> <summary><h2>Intended Use & Limitations</h2></summary> <div class="info-card"> <div class="card-content"> <ul> <li><strong>Intended Use:</strong> The primary purpose of this model is for creative and roleplaying within the <em>Wings of Fire</em> universe. However, user feedback indicates it is also highly effective for general-purpose roleplaying.</li> <li><strong>Limitations & Quirks:</strong> <ul> <li><strong>Experimental Features:</strong> The custom <code><choices></choices></code> tag functionality rarely works. The model may occasionally attempt to use it, but users should not expect reliable structured output in this format. Despite this, the model's overall quality remains very high.</li> <li>Performance on tasks outside of its training domain (general knowledge, coding, etc.) is not guaranteed and will likely be poor.</li> <li><strong>Versatility:</strong> While specifically tuned for <em>Wings of Fire</em>, the model has proven to be very capable of performing normal roleplay with other settings and characters.</li> <li>The model may "hallucinate" or generate plausible but non-canonical information, especially when pushed outside the established "what-if" scenarios.</li> <li><strong>Content:</strong> The training data includes mature and darker themes from the <em>Wings of Fire</em> series, such as conflict, character death, and moral ambiguity. The model is capable of generating content reflecting these themes. As always, it is up to the user what they do with it.</li> <li><strong>Formatting:</strong> Training data was cleaned to remove narrative artifacts like <code>**scene transitions**</code>. The model should now produce cleaner prose.</li> <li><strong>Safety:</strong> This model has not undergone additional safety alignment beyond what was included in its `M3.2-36b` base model. Standard responsible AI practices should be followed.</li> </ul> </li> </ul> </div> </div> </details> </div> <div class="section-container"> <details> <summary><h2>Acknowledgements</h2></summary> <div class="info-card"> <div class="card-content"> <ul> <li>Credit to Mistral AI and CrucibleLab-TG for the powerful <code>M3.2-36b</code> base model.</li> <li>Credit to Google for the Gemini Pro model, used in dataset generation.</li> <li>Credit to Evan Armstrong for <a href="https://github.com/e-p-armstrong/augmentoolkit" target="_blank">Augmentoolkit</a>, an invaluable tool for dataset creation.</li> </ul> </div> </div> </details> </div> </div> </div>