Magistry-24B-v1.1-mlx-4bit-gs32

beezu/Magistry-24B-v1.1-mlx-4Bit-gs32

This model beezu/Magistry-24B-v1.1-mlx-4Bit-gs32 was converted to MLX format from sophosympatheia/Magistry-24B-v1.1 using mlx-lm version 0.31.2.

About this quant

Uses group size of 32.

Original Model Card

<span style="color: #C9A84C; text-align: center; display: block; font-size: 2.5rem; margin-bottom: 0.3em; text-shadow: 0 0 10px rgba(201, 168, 76, 0.3);">Magistry-24B-v1.1</span>

<p style="text-align: center; color: #A89870; letter-spacing: 0.2em; text-transform: uppercase; font-size: 0.85rem; margin-top: 0; margin-bottom: 1.5em;">A Royal Merge &nbsp;·&nbsp; 24B &nbsp;·&nbsp; Apache 2.0</p> <div style="background: linear-gradient(to bottom, rgba(201, 168, 76, 0.07), rgba(26, 22, 16, 0.9)); border: 1px solid rgba(201, 168, 76, 0.35); border-radius: 8px; padding: 25px; margin-bottom: 30px; position: relative; overflow: hidden; box-shadow: 0 4px 20px rgba(0, 0, 0, 0.5);"> <div style="position: absolute; top: 0; left: 0; width: 100%; height: 5px; background: linear-gradient(90deg, #A07830, #C9A84C, #E8C96A, #C9A84C, #A07830); opacity: 0.9;"></div> <img src="https://i.imgur.com/q3KTR0b.png" alt="StrawberryLemonade" style="width: 80%; min-width: 400px; display: block; margin: auto; border-radius: 8px; margin-bottom: 1.5em;"> <p style="color: #D4C9A8; margin-bottom: 1em;">Please see my model card for <a href="https://huggingface.co/sophosympatheia/Magistry-24B-v1.0" style="color: #C9A84C; text-decoration: none; border-bottom: 1px dotted rgba(201, 168, 76, 0.4);">sophosympatheia/Magistry-24B-v1.0</a> for background information on this line of models.</p> <p style="color: #D4C9A8; margin-bottom: 0;">Magistry-24B-v1.1 was the result of my attempts to improve on the coherency and handling of v1.0 without compromising what makes it fun. After performing some dark rites in my laboratory, I think I managed to produce a worthy successor. This version of Magistry feels like a different beast. It is hard to say whether it is truly "smarter" than v1.0, but it at least writes like it wants to be seen that way. Is it? Test it for yourself and let me know. It didn't lose any of its pizzazz, that's for sure. If anything, I think Devstral somehow made this version even more creative and entertaining.</p> </div>

<span style="color: #C9A84C; font-size: 1.8rem; border-bottom: 1px solid rgba(201, 168, 76, 0.3); padding-bottom: 0.3em; display: block;">Known Issues</span>

<div style="background-color: #1E1A10; border: 1px solid rgba(201, 168, 76, 0.3); border-radius: 8px; padding: 25px; margin-bottom: 30px; position: relative; overflow: hidden; box-shadow: 0 4px 20px rgba(0, 0, 0, 0.5);"> <div style="position: absolute; top: 0; left: 0; width: 100%; height: 4px; background: linear-gradient(90deg, #A07830, #C9A84C, #E8C96A, #C9A84C, #A07830); opacity: 0.7;"></div> <p style="color: #D4C9A8; margin-bottom: 0;">This model is fun, but it will struggle with details of logical/physical continuity at times — which is probably inescapable for a 24B model. Rerolling the output might fix it, or you might have to help it out by providing explicit instructions or 'reminders' so it doesn't get confused. It should at least outperform v1.0 in this area.</p> <p style="color: #D4C9A8; margin-bottom: 0;">Thinking can also help this model with continuity and planning, but don't expect miracles. If you force its reply to start with &lt;think&gt; and give it some basic instructions, it can totally do it. I find &lt;think&gt;&lt;/think&gt; tags work better than Mistral's [THINK][/THINK] tags.</p> <div style="padding: 15px;"> <pre style="background-color: rgba(0, 0, 0, 0.3); border-radius: 6px; padding: 15px; overflow-x: auto; border: 1px solid rgba(201, 168, 76, 0.2); margin: 0;"><code style="font-family: 'Courier New', Courier, monospace; font-size: 0.85em; color: #c8bfa0;">&lt;think&gt; First draft your thinking process (inner monologue) until you arrive at a response. Your thinking process must follow the template below: Your thoughts or/and draft, like working through an exercise on scratch paper. Be as casual and as long as you want until you are confident to generate the response. Responses must develop logically from past context. THINKING: </code></pre> </div> </div>

<span style="color: #C9A84C; font-size: 1.8rem; border-bottom: 1px solid rgba(201, 168, 76, 0.3); padding-bottom: 0.3em; display: block;">Sampler Tips</span>

<div style="background-color: #1E1A10; border: 1px solid rgba(201, 168, 76, 0.3); border-radius: 8px; padding: 25px; margin-bottom: 30px; position: relative; overflow: hidden; box-shadow: 0 4px 20px rgba(0, 0, 0, 0.5);"> <div style="position: absolute; top: 0; left: 0; width: 100%; height: 4px; background: linear-gradient(90deg, #A07830, #C9A84C, #E8C96A, #C9A84C, #A07830); opacity: 0.7;"></div> <p style="color: #D4C9A8; margin-bottom: 1em;">You can use the master import JSON in this repo (Magistry_SillyTavern_Master_Import.json) to deploy the conservative sampler settings below, which are likely to be compatible with more backend/frontend combos. I recommend using these values as a <strong style="color: #E8C96A;">starting point</strong> for your own experiments. It's not like the model falls apart if you deviate from these settings, but they should be a reliable starting point for most creative tasks.</p> <h3 style="color: #EDE8DC; font-size: 1.2rem; margin-top: 1.5em; margin-bottom: 0.8em;">Conservative Settings</h3> <p style="color: #D4C9A8; margin-bottom: 1em;">Run these settings as a starting point if you care more about accuracy than flair. If you can't use Top-N σ, you can safely drop it for ubiquitous samplers like Top-P and Top-K, and then tune those settings plus Temperature and Min-P to get virtually the same effect.</p> <div style="display: grid; grid-template-columns: repeat(auto-fit, minmax(130px, 1fr)); gap: 0.8rem; margin-bottom: 1.5em;"> <div style="background: rgba(201, 168, 76, 0.07); border: 1px solid rgba(201, 168, 76, 0.35); border-radius: 5px; padding: 0.7rem 1rem; text-align: center;"> <span style="display: block; font-size: 0.7rem; text-transform: uppercase; letter-spacing: 0.12em; color: #C9A84C; margin-bottom: 0.25rem;">Temp</span> <span style="display: block; font-size: 1.05rem; font-weight: 700; color: #F5F0E8; font-family: 'Courier New', monospace;">0.7</span> </div> <div style="background: rgba(201, 168, 76, 0.07); border: 1px solid rgba(201, 168, 76, 0.35); border-radius: 5px; padding: 0.7rem 1rem; text-align: center;"> <span style="display: block; font-size: 0.7rem; text-transform: uppercase; letter-spacing: 0.12em; color: #C9A84C; margin-bottom: 0.25rem;">Min-P</span> <span style="display: block; font-size: 1.05rem; font-weight: 700; color: #F5F0E8; font-family: 'Courier New', monospace;">0.05</span> </div> <div style="background: rgba(201, 168, 76, 0.07); border: 1px solid rgba(201, 168, 76, 0.35); border-radius: 5px; padding: 0.7rem 1rem; text-align: center;"> <span style="display: block; font-size: 0.7rem; text-transform: uppercase; letter-spacing: 0.12em; color: #C9A84C; margin-bottom: 0.25rem;">Top-N σ</span> <span style="display: block; font-size: 1.05rem; font-weight: 700; color: #F5F0E8; font-family: 'Courier New', monospace;">0.75</span> </div> <div style="background: rgba(201, 168, 76, 0.07); border: 1px solid rgba(201, 168, 76, 0.35); border-radius: 5px; padding: 0.7rem 1rem; text-align: center;"> <span style="display: block; font-size: 0.7rem; text-transform: uppercase; letter-spacing: 0.12em; color: #C9A84C; margin-bottom: 0.25rem;">DRY Mult.</span> <span style="display: block; font-size: 1.05rem; font-weight: 700; color: #F5F0E8; font-family: 'Courier New', monospace;">0.8</span> </div> <div style="background: rgba(201, 168, 76, 0.07); border: 1px solid rgba(201, 168, 76, 0.35); border-radius: 5px; padding: 0.7rem 1rem; text-align: center;"> <span style="display: block; font-size: 0.7rem; text-transform: uppercase; letter-spacing: 0.12em; color: #C9A84C; margin-bottom: 0.25rem;">DRY Base</span> <span style="display: block; font-size: 1.05rem; font-weight: 700; color: #F5F0E8; font-family: 'Courier New', monospace;">1.8</span> </div> </div> <h3 style="color: #EDE8DC; font-size: 1.2rem; margin-top: 1.5em; margin-bottom: 0.8em;">Wild Settings</h3> <p style="color: #D4C9A8; margin-bottom: 1em;">Run these settings as a starting point if you care more about flair and fun than accuracy. You will have to reroll more often and deal with shenanigans, but they'll be entertaining most of the time. These Adaptive-P settings differ from what is usually recommended, but trust me on this: it's fun with this model and works fine for simple scenarios and conversations. The low Adaptive-P Decay setting frees the model to "fishtail" between wild token choices and conservative token choices in a way that mostly keeps things on track while unleashing creativity. If you need stability for complex scenarios, then run either the Conservative Settings above or the Balanced Settings below.</p> <div style="display: grid; grid-template-columns: repeat(auto-fit, minmax(130px, 1fr)); gap: 0.8rem; margin-bottom: 0.5em;"> <div style="background: rgba(201, 168, 76, 0.07); border: 1px solid rgba(201, 168, 76, 0.35); border-radius: 5px; padding: 0.7rem 1rem; text-align: center;"> <span style="display: block; font-size: 0.7rem; text-transform: uppercase; letter-spacing: 0.12em; color: #C9A84C; margin-bottom: 0.25rem;">Temp</span> <span style="display: block; font-size: 1.05rem; font-weight: 700; color: #F5F0E8; font-family: 'Courier New', monospace;">0.9</span> </div> <div style="background: rgba(201, 168, 76, 0.07); border: 1px solid rgba(201, 168, 76, 0.35); border-radius: 5px; padding: 0.7rem 1rem; text-align: center;"> <span style="display: block; font-size: 0.7rem; text-transform: uppercase; letter-spacing: 0.12em; color: #C9A84C; margin-bottom: 0.25rem;">Min-P</span> <span style="display: block; font-size: 1.05rem; font-weight: 700; color: #F5F0E8; font-family: 'Courier New', monospace;">0.05</span> </div> <div style="background: rgba(201, 168, 76, 0.07); border: 1px solid rgba(201, 168, 76, 0.35); border-radius: 5px; padding: 0.7rem 1rem; text-align: center;"> <span style="display: block; font-size: 0.7rem; text-transform: uppercase; letter-spacing: 0.12em; color: #C9A84C; margin-bottom: 0.25rem;">Adaptive-P Target</span> <span style="display: block; font-size: 1.05rem; font-weight: 700; color: #F5F0E8; font-family: 'Courier New', monospace;">0.35</span> </div> <div style="background: rgba(201, 168, 76, 0.07); border: 1px solid rgba(201, 168, 76, 0.35); border-radius: 5px; padding: 0.7rem 1rem; text-align: center;"> <span style="display: block; font-size: 0.7rem; text-transform: uppercase; letter-spacing: 0.12em; color: #C9A84C; margin-bottom: 0.25rem;">Adaptive-P Decay</span> <span style="display: block; font-size: 1.05rem; font-weight: 700; color: #F5F0E8; font-family: 'Courier New', monospace;">0.45</span> </div> </div> <h3 style="color: #EDE8DC; font-size: 1.2rem; margin-top: 1.5em; margin-bottom: 0.8em;">Balanced Settings</h3> <p style="color: #D4C9A8; margin-bottom: 1em;">Run these settings as a starting point if you want a reasonably stable balance of flair and accuracy. Adaptive-P seems to work wonders with this model, and you can tune the parameters to get more or less predictable results. (<a href="https://github.com/MrJackSpade/adaptive-p-docs/blob/main/sections/06_parameters.md">See the creator's GitHub docs for more information on Adaptive-P parameters</a>.) If you need more stability, lower Temperature and increase Adaptive-P Target. If you want more creativity, lower Adaptive-P Target. Tune Adaptive-P Decay last since the default 0.9 is recommended and stable.</p> <div style="display: grid; grid-template-columns: repeat(auto-fit, minmax(130px, 1fr)); gap: 0.8rem; margin-bottom: 0.5em;"> <div style="background: rgba(201, 168, 76, 0.07); border: 1px solid rgba(201, 168, 76, 0.35); border-radius: 5px; padding: 0.7rem 1rem; text-align: center;"> <span style="display: block; font-size: 0.7rem; text-transform: uppercase; letter-spacing: 0.12em; color: #C9A84C; margin-bottom: 0.25rem;">Temp</span> <span style="display: block; font-size: 1.05rem; font-weight: 700; color: #F5F0E8; font-family: 'Courier New', monospace;">1.0</span> </div> <div style="background: rgba(201, 168, 76, 0.07); border: 1px solid rgba(201, 168, 76, 0.35); border-radius: 5px; padding: 0.7rem 1rem; text-align: center;"> <span style="display: block; font-size: 0.7rem; text-transform: uppercase; letter-spacing: 0.12em; color: #C9A84C; margin-bottom: 0.25rem;">Min-P</span> <span style="display: block; font-size: 1.05rem; font-weight: 700; color: #F5F0E8; font-family: 'Courier New', monospace;">0.05</span> </div> <div style="background: rgba(201, 168, 76, 0.07); border: 1px solid rgba(201, 168, 76, 0.35); border-radius: 5px; padding: 0.7rem 1rem; text-align: center;"> <span style="display: block; font-size: 0.7rem; text-transform: uppercase; letter-spacing: 0.12em; color: #C9A84C; margin-bottom: 0.25rem;">Adaptive-P Target</span> <span style="display: block; font-size: 1.05rem; font-weight: 700; color: #F5F0E8; font-family: 'Courier New', monospace;">0.6</span> </div> <div style="background: rgba(201, 168, 76, 0.07); border: 1px solid rgba(201, 168, 76, 0.35); border-radius: 5px; padding: 0.7rem 1rem; text-align: center;"> <span style="display: block; font-size: 0.7rem; text-transform: uppercase; letter-spacing: 0.12em; color: #C9A84C; margin-bottom: 0.25rem;">Adaptive-P Decay</span> <span style="display: block; font-size: 1.05rem; font-weight: 700; color: #F5F0E8; font-family: 'Courier New', monospace;">0.9</span> </div> </div> </div>

<span style="color: #C9A84C; font-size: 1.8rem; border-bottom: 1px solid rgba(201, 168, 76, 0.3); padding-bottom: 0.3em; display: block;">Prompting Tips</span>

<div style="background-color: #1E1A10; border: 1px solid rgba(201, 168, 76, 0.3); border-radius: 8px; padding: 25px; margin-bottom: 30px; position: relative; overflow: hidden; box-shadow: 0 4px 20px rgba(0, 0, 0, 0.5);"> <div style="position: absolute; top: 0; left: 0; width: 100%; height: 4px; background: linear-gradient(90deg, #A07830, #C9A84C, #E8C96A, #C9A84C, #A07830); opacity: 0.7;"></div> <p style="color: #D4C9A8; margin-bottom: 0;">You can download the <code style="background-color: rgba(201, 168, 76, 0.1); border-radius: 3px; padding: 0.1em 0.35em; color: #E8C96A; font-family: 'Courier New', monospace;">Magistry_SillyTavern_Master_Import.json</code> file from this repo and import it directly into SillyTavern to get system prompt, chat template, and sampler settings all in one go.</p> </div>

<span style="color: #C9A84C; font-size: 1.8rem; border-bottom: 1px solid rgba(201, 168, 76, 0.3); padding-bottom: 0.3em; display: block;">Donations</span>

<div style="display: flex; align-items: center; justify-content: center; flex-direction: column; gap: 10px; padding: 20px; background: linear-gradient(to bottom, rgba(201, 168, 76, 0.07), rgba(26, 22, 16, 0.9)); border-radius: 8px; border: 1px solid rgba(201, 168, 76, 0.35); margin-top: 10px; margin-bottom: 30px;"> <a href="https://ko-fi.com/sophosympatheia"> <img src="https://i.imgur.com/LySwHVd.png" alt="Donations" style="max-width: 200px; width: 100%;"> </a> <p style="color: #D4C9A8; margin-bottom: 0.5em; text-align: center;">If you feel like saying thanks with a donation, <a href="https://ko-fi.com/sophosympatheia" style="display: inline-block; background: linear-gradient(45deg, #A07830, #C9A84C, #E8C96A); color: #1A1610; padding: 10px 20px; border-radius: 6px; font-weight: 700; letter-spacing: 0.5px; border: none; box-shadow: 0 4px 15px rgba(201, 168, 76, 0.3); text-decoration: none !important;">I'm on Ko-Fi</a></p> </div>

<span style="color: #C9A84C; font-size: 1.8rem; border-bottom: 1px solid rgba(201, 168, 76, 0.3); padding-bottom: 0.3em; display: block;">Quantizations</span>

<div style="background-color: #1E1A10; border: 1px solid rgba(201, 168, 76, 0.3); border-radius: 8px; padding: 25px; margin-bottom: 30px; position: relative; overflow: hidden; box-shadow: 0 4px 20px rgba(0, 0, 0, 0.5);"> <div style="position: absolute; top: 0; left: 0; width: 100%; height: 4px; background: linear-gradient(90deg, #A07830, #C9A84C, #E8C96A, #C9A84C, #A07830); opacity: 0.7;"></div> <p style="color: #A89870; font-style: italic; margin-bottom: 0;">Please see the sidebar of the model card where a link to quantizations can be found, or <a href="https://huggingface.co/models?other=base_model:quantized:sophosympatheia/Magistry-24B-v1.1">click here for the list of them</a>.</p> </div>

<span style="color: #C9A84C; font-size: 1.8rem; border-bottom: 1px solid rgba(201, 168, 76, 0.3); padding-bottom: 0.3em; display: block;">License</span>

<div style="background-color: #1E1A10; border: 1px solid rgba(201, 168, 76, 0.3); border-radius: 8px; padding: 25px; margin-bottom: 30px; position: relative; overflow: hidden; box-shadow: 0 4px 20px rgba(0, 0, 0, 0.5);"> <div style="position: absolute; top: 0; left: 0; width: 100%; height: 4px; background: linear-gradient(90deg, #A07830, #C9A84C, #E8C96A, #C9A84C, #A07830); opacity: 0.7;"></div> <p style="color: #D4C9A8; margin-bottom: 0;">Apache 2.0, inherited down from Magistral.</p> </div>

<span style="color: #C9A84C; font-size: 1.8rem; border-bottom: 1px solid rgba(201, 168, 76, 0.3); padding-bottom: 0.3em; display: block;">Merge Details</span>

<div style="background-color: #1E1A10; border: 1px solid rgba(201, 168, 76, 0.3); border-radius: 8px; padding: 25px; margin-bottom: 30px; position: relative; overflow: hidden; box-shadow: 0 4px 20px rgba(0, 0, 0, 0.5);"> <div style="position: absolute; top: 0; left: 0; width: 100%; height: 4px; background: linear-gradient(90deg, #A07830, #C9A84C, #E8C96A, #C9A84C, #A07830); opacity: 0.7;"></div> <p style="color: #D4C9A8; margin-bottom: 1em;">This is a merge of pre-trained language models created using <a href="https://github.com/cg123/mergekit" style="color: #C9A84C; text-decoration: none; border-bottom: 1px dotted rgba(201, 168, 76, 0.4);">mergekit</a>.</p> <h3 style="color: #EDE8DC; font-size: 1.2rem; margin-top: 0.5em; margin-bottom: 0.5em;">Merge Method</h3> <p style="color: #D4C9A8; margin-bottom: 1em;">This model was merged using the <a href="https://arxiv.org/abs/2406.11617" style="color: #C9A84C; text-decoration: none; border-bottom: 1px dotted rgba(201, 168, 76, 0.4);">DELLA</a> merge method, using <a href="https://huggingface.co/Darkhn/Magistral-2509-24B-Text-Only" style="color: #C9A84C; text-decoration: none; border-bottom: 1px dotted rgba(201, 168, 76, 0.4);">Darkhn/Magistral-2509-24B-Text-Only</a> as a base.</p> <h3 style="color: #EDE8DC; font-size: 1.2rem; margin-top: 1.5em; margin-bottom: 0.5em;">Models Merged</h3> <p style="color: #D4C9A8; margin-bottom: 0.5em;">The following models were included in the merge:</p> <ul style="color: #D4C9A8; margin-bottom: 1em; padding-left: 20px;"> <li style="margin-bottom: 0.5em;"><a href="https://huggingface.co/Casual-Autopsy/Maginum-Cydoms-24B" style="color: #C9A84C; text-decoration: none; border-bottom: 1px dotted rgba(201, 168, 76, 0.4);">Casual-Autopsy/Maginum-Cydoms-24B</a></li> <li style="margin-bottom: 0.5em;"><a href="https://huggingface.co/DarkArtsForge/Magistaroth-24B-v1" style="color: #C9A84C; text-decoration: none; border-bottom: 1px dotted rgba(201, 168, 76, 0.4);">DarkArtsForge/Magistaroth-24B-v1</a></li> <li style="margin-bottom: 0.5em;"><a href="https://huggingface.co/huihui-ai/Huihui-Devstral-Small-2-24B-Instruct-2512-abliterated" style="color: #C9A84C; text-decoration: none; border-bottom: 1px dotted rgba(201, 168, 76, 0.4);">huihui-ai/Huihui-Devstral-Small-2-24B-Instruct-2512-abliterated</a> - I had to process this model before using it in the merge, plus get mergekit to work with it by backporting ministral3 support into transformers v4.</li> </ul> <details style="margin-bottom: 0; background-color: #111009; border-radius: 8px; overflow: hidden; border: 1px solid rgba(201, 168, 76, 0.25);"> <summary style="padding: 12px 15px; cursor: pointer; background-color: #1E1A10; border-bottom: 1px solid rgba(201, 168, 76, 0.25); font-weight: 600; color: #C9A84C; display: flex; align-items: center;">Configuration YAML</summary> <div style="padding: 15px;"> <pre style="background-color: rgba(0, 0, 0, 0.3); border-radius: 6px; padding: 15px; overflow-x: auto; border: 1px solid rgba(201, 168, 76, 0.2); margin: 0;"><code style="font-family: 'Courier New', Courier, monospace; font-size: 0.85em; color: #c8bfa0;"> models: - model: Darkhn/Magistral-2509-24B-Text-Only # base model — no parameters needed - model: Casual-Autopsy/Maginum-Cydoms-24B parameters: weight: - filter: "self_attn" value: 0.55 # pull attn closer to base — preserves magistral's reasoning - filter: "mlp" value: 0.8 # keep mlp weight — creative energy - value: 0.8 # fallback for embed/norm/lm_head density: 0.9 epsilon: 0.099 - model: DarkArtsForge/Magistaroth-24B-v1 parameters: weight: - filter: "self_attn" value: 0.55 # pull attn closer to base — preserves magistral's reasoning - filter: "mlp" value: 0.8 # keep mlp weight — creative energy - value: 0.8 # fallback for embed/norm/lm_head density: 0.9 epsilon: 0.099 - model: huihui-ai/Huihui-Devstral-Small-2-24B-Instruct-2512-abliterated # Note: this cannot be merged directly without significant pre-processing parameters: weight: 0.3 density: 0.9 epsilon: 0.099

merge_method: della base_model: /home/llm/mergequant/models/BASE/Darkhn_Magistral-2509-24B-Text-Only

parameters: lambda: 1.0 normalize: false

tokenizer: source: union chat_template: auto dtype: float32 out_dtype: bfloat16 </code></pre> </div>

</details> </div>