beezu/Glistening-Gem-31B-v1.0-mlx-4bit-gs32
This model beezu/Glistening-Gem-31B-v1.0-mlx-4bit-gs32 was converted to MLX format from sophosympatheia/Glistening-Gem-31B-v1.0 using mlx-lm version 0.31.3.
About this quant
Uses group size of 32.
Original Model Card
<span style="color: #5BA4CF; text-align: center; display: block; font-size: 2.5rem; margin-bottom: 0.3em; text-shadow: 0 0 10px rgba(91, 164, 207, 0.3);">Glistening-Gem-31B-v1.0</span>
<p style="text-align: center; color: #7A9BB5; letter-spacing: 0.2em; text-transform: uppercase; font-size: 0.85rem; margin-top: 0; margin-bottom: 1.5em;">An Experimental Merge · 31B · Apache 2.0</p>
<div style="background: linear-gradient(to bottom, rgba(91, 164, 207, 0.07), rgba(14, 20, 28, 0.9)); border: 1px solid rgba(91, 164, 207, 0.35); border-radius: 8px; padding: 25px; margin-bottom: 30px; position: relative; overflow: hidden; box-shadow: 0 4px 20px rgba(0, 0, 0, 0.5);">
<div style="position: absolute; top: 0; left: 0; width: 100%; height: 5px; background: linear-gradient(90deg, #2A6B8C, #5BA4CF, #8BCAE6, #5BA4CF, #2A6B8C); opacity: 0.9;"></div>
<p style="color: #B8D4E8; margin-bottom: 1em;">This is an experimental merge of <a href="https://huggingface.co/BeaverAI/Artemis-31B-v1h-GGUF" style="color: #5BA4CF; text-decoration: none; border-bottom: 1px dotted rgba(91, 164, 207, 0.4);">BeaverAI/Artemis-31B-v1h-GGUF</a>, <a href="https://huggingface.co/zerofata/G4-MeroMero-31B" style="color: #5BA4CF; text-decoration: none; border-bottom: 1px dotted rgba(91, 164, 207, 0.4);">zerofata/G4-MeroMero-31B</a>, and <a href="https://huggingface.co/llmfan46/gemma-4-Ortenzya-The-Creative-Wordsmith-31B-it-uncensored-heretic" style="color: #5BA4CF; text-decoration: none; border-bottom: 1px dotted rgba(91, 164, 207, 0.4);">llmfan46/gemma-4-Ortenzya-The-Creative-Wordsmith-31B-it-uncensored-heretic</a>.</p>
<p style="color: #B8D4E8; margin-bottom: 1em;">TheDrummer (creator of Artemis) has not released the full precision FP16 weights for the Artemis v1h model, so I had to get creative with a Q8_0 GGUF file to produce this merge. There is some precision loss in doing that, but Q8_0 is already pretty close to FP16 in practice, and I think the noise effectively washes out during the merge process.</p>
<p style="color: #B8D4E8; margin-bottom: 1em;">This merge recipe grew out of my initial experiment with <a href="https://huggingface.co/sophosympatheia/Mero-Artemis-31B-v0.3.1" style="color: #5BA4CF; text-decoration: none; border-bottom: 1px dotted rgba(91, 164, 207, 0.4);">sophosympatheia/Mero-Artemis-31B-v0.3.1</a> which was positively received by the community.</p>
<p style="color: #B8D4E8; margin-bottom: 0;">Glimmer has more diversity in its prose and style compared to Mero-Artemis thanks to Ortenzya's influence. However, Mero-Artemis is overall more stable.</p>
</div>
<span style="color: #5BA4CF; font-size: 1.8rem; border-bottom: 1px solid rgba(91, 164, 207, 0.3); padding-bottom: 0.3em; display: block;">Known Issues</span>
<div style="background-color: #121A24; border: 1px solid rgba(91, 164, 207, 0.3); border-radius: 8px; padding: 25px; margin-bottom: 30px; position: relative; overflow: hidden; box-shadow: 0 4px 20px rgba(0, 0, 0, 0.5);">
<div style="position: absolute; top: 0; left: 0; width: 100%; height: 4px; background: linear-gradient(90deg, #2A6B8C, #5BA4CF, #8BCAE6, #5BA4CF, #2A6B8C); opacity: 0.7;"></div>
<p style="color: #B8D4E8; margin-bottom: 1em;">This model requires conservative sampler settings to maintain coherence as the conversation grows in context length. It will produce erroneous outputs if you run it with "aggressive" sampler settings for creativity, so dial it back if you encounter misspellings, grammatical errors, or illogical outputs.</p>
<p style="color: #B8D4E8; margin-bottom: 1em;">I'd say this model is right on the cusp of instability, but that puts it in an interesting position.</p>
<p style="color: #E8C96A; font-weight: bold; margin-bottom: 0; font-size: 1.1rem;">IF YOU GET WEIRD OUTPUTS, TURN DOWN THE HEAT</p>
</div>
<span style="color: #5BA4CF; font-size: 1.8rem; border-bottom: 1px solid rgba(91, 164, 207, 0.3); padding-bottom: 0.3em; display: block;">Sampler Tips</span>
<div style="background-color: #121A24; border: 1px solid rgba(91, 164, 207, 0.3); border-radius: 8px; padding: 25px; margin-bottom: 30px; position: relative; overflow: hidden; box-shadow: 0 4px 20px rgba(0, 0, 0, 0.5);">
<div style="position: absolute; top: 0; left: 0; width: 100%; height: 4px; background: linear-gradient(90deg, #2A6B8C, #5BA4CF, #8BCAE6, #5BA4CF, #2A6B8C); opacity: 0.7;"></div>
<p style="color: #B8D4E8; margin-bottom: 1em;">You can use the master import JSON in this repo (Glimmer_SillyTavern_Master_Import.json) to deploy the conservative sampler settings below, which are likely to be compatible with more backend/frontend combos. I recommend using these values as a <strong style="color: #8BCAE6;">starting point</strong> for your own experiments. It's not like the model falls apart if you deviate from these settings, but they should be a reliable starting point for most creative tasks.</p>
<h3 style="color: #D4E8F5; font-size: 1.2rem; margin-top: 1.5em; margin-bottom: 0.8em;">Conservative Settings</h3>
<p style="color: #B8D4E8; margin-bottom: 1em;">Run these settings as a starting point. Raise temperature and lower Min-P if you want more creativity.</p>
<div style="display: grid; grid-template-columns: repeat(auto-fit, minmax(130px, 1fr)); gap: 0.8rem; margin-bottom: 1.5em;">
<div style="background: rgba(91, 164, 207, 0.07); border: 1px solid rgba(91, 164, 207, 0.35); border-radius: 5px; padding: 0.7rem 1rem; text-align: center;">
<span style="display: block; font-size: 0.7rem; text-transform: uppercase; letter-spacing: 0.12em; color: #5BA4CF; margin-bottom: 0.25rem;">Temp</span>
<span style="display: block; font-size: 1.05rem; font-weight: 700; color: #E8F0F8; font-family: 'Courier New', monospace;">0.7</span>
</div>
<div style="background: rgba(91, 164, 207, 0.07); border: 1px solid rgba(91, 164, 207, 0.35); border-radius: 5px; padding: 0.7rem 1rem; text-align: center;">
<span style="display: block; font-size: 0.7rem; text-transform: uppercase; letter-spacing: 0.12em; color: #5BA4CF; margin-bottom: 0.25rem;">Min-P</span>
<span style="display: block; font-size: 1.05rem; font-weight: 700; color: #E8F0F8; font-family: 'Courier New', monospace;">0.2</span>
</div>
<div style="background: rgba(91, 164, 207, 0.07); border: 1px solid rgba(91, 164, 207, 0.35); border-radius: 5px; padding: 0.7rem 1rem; text-align: center;">
<span style="display: block; font-size: 0.7rem; text-transform: uppercase; letter-spacing: 0.12em; color: #5BA4CF; margin-bottom: 0.25rem;">Adaptive-P Target</span>
<span style="display: block; font-size: 1.05rem; font-weight: 700; color: #E8F0F8; font-family: 'Courier New', monospace;">0.6</span>
</div>
<div style="background: rgba(91, 164, 207, 0.07); border: 1px solid rgba(91, 164, 207, 0.35); border-radius: 5px; padding: 0.7rem 1rem; text-align: center;">
<span style="display: block; font-size: 0.7rem; text-transform: uppercase; letter-spacing: 0.12em; color: #5BA4CF; margin-bottom: 0.25rem;">Adaptive-P Decay</span>
<span style="display: block; font-size: 1.05rem; font-weight: 700; color: #E8F0F8; font-family: 'Courier New', monospace;">0.5</span>
</div>
<div style="background: rgba(91, 164, 207, 0.07); border: 1px solid rgba(91, 164, 207, 0.35); border-radius: 5px; padding: 0.7rem 1rem; text-align: center;">
<span style="display: block; font-size: 0.7rem; text-transform: uppercase; letter-spacing: 0.12em; color: #5BA4CF; margin-bottom: 0.25rem;">DRY Mult.</span>
<span style="display: block; font-size: 1.05rem; font-weight: 700; color: #E8F0F8; font-family: 'Courier New', monospace;">0.8</span>
</div>
<div style="background: rgba(91, 164, 207, 0.07); border: 1px solid rgba(91, 164, 207, 0.35); border-radius: 5px; padding: 0.7rem 1rem; text-align: center;">
<span style="display: block; font-size: 0.7rem; text-transform: uppercase; letter-spacing: 0.12em; color: #5BA4CF; margin-bottom: 0.25rem;">DRY Base</span>
<span style="display: block; font-size: 1.05rem; font-weight: 700; color: #E8F0F8; font-family: 'Courier New', monospace;">1.8</span>
</div>
</div>
</div>
<span style="color: #5BA4CF; font-size: 1.8rem; border-bottom: 1px solid rgba(91, 164, 207, 0.3); padding-bottom: 0.3em; display: block;">Prompting Tips</span>
<div style="background-color: #121A24; border: 1px solid rgba(91, 164, 207, 0.3); border-radius: 8px; padding: 25px; margin-bottom: 30px; position: relative; overflow: hidden; box-shadow: 0 4px 20px rgba(0, 0, 0, 0.5);">
<div style="position: absolute; top: 0; left: 0; width: 100%; height: 4px; background: linear-gradient(90deg, #2A6B8C, #5BA4CF, #8BCAE6, #5BA4CF, #2A6B8C); opacity: 0.7;"></div>
<p style="color: #B8D4E8; margin-bottom: 0;">You can download the <code style="background-color: rgba(91, 164, 207, 0.1); border-radius: 3px; padding: 0.1em 0.35em; color: #8BCAE6; font-family: 'Courier New', monospace;">Glistening-Gem_SillyTavern_Master_Import.json</code> file from this repo and import it directly into SillyTavern to get system prompt, chat template, and sampler settings all in one go.</p>
</div>
<span style="color: #5BA4CF; font-size: 1.8rem; border-bottom: 1px solid rgba(91, 164, 207, 0.3); padding-bottom: 0.3em; display: block;">Donations</span>
<div style="display: flex; align-items: center; justify-content: center; flex-direction: column; gap: 10px; padding: 20px; background: linear-gradient(to bottom, rgba(91, 164, 207, 0.07), rgba(14, 20, 28, 0.9)); border-radius: 8px; border: 1px solid rgba(91, 164, 207, 0.35); margin-top: 10px; margin-bottom: 30px;">
<a href="https://ko-fi.com/sophosympatheia">
<img src="https://i.imgur.com/LySwHVd.png" alt="Donations" style="max-width: 200px; width: 100%;">
</a>
<p style="color: #B8D4E8; margin-bottom: 0.5em; text-align: center;">If you feel like saying thanks with a donation, <a href="https://ko-fi.com/sophosympatheia" style="display: inline-block; background: linear-gradient(45deg, #2A6B8C, #5BA4CF, #8BCAE6); color: #0E141C; padding: 10px 20px; border-radius: 6px; font-weight: 700; letter-spacing: 0.5px; border: none; box-shadow: 0 4px 15px rgba(91, 164, 207, 0.3); text-decoration: none !important;">I'm on Ko-Fi</a></p>
</div>
<span style="color: #5BA4CF; font-size: 1.8rem; border-bottom: 1px solid rgba(91, 164, 207, 0.3); padding-bottom: 0.3em; display: block;">Quantizations</span>
<div style="background-color: #121A24; border: 1px solid rgba(91, 164, 207, 0.3); border-radius: 8px; padding: 25px; margin-bottom: 30px; position: relative; overflow: hidden; box-shadow: 0 4px 20px rgba(0, 0, 0, 0.5);">
<div style="position: absolute; top: 0; left: 0; width: 100%; height: 4px; background: linear-gradient(90deg, #2A6B8C, #5BA4CF, #8BCAE6, #5BA4CF, #2A6B8C); opacity: 0.7;"></div>
<p style="color: #7A9BB5; font-style: italic; margin-bottom: 0;">Please see the sidebar of the model card where a link to quantizations can be found, or <a href="https://huggingface.co/models?other=base_model:quantized:sophosympatheia/Glistening-Gem-31B-v1.0" style="color: #5BA4CF; text-decoration: none; border-bottom: 1px dotted rgba(91, 164, 207, 0.4);">click here for the list of them</a>.</p>
</div>
<span style="color: #5BA4CF; font-size: 1.8rem; border-bottom: 1px solid rgba(91, 164, 207, 0.3); padding-bottom: 0.3em; display: block;">License</span>
<div style="background-color: #121A24; border: 1px solid rgba(91, 164, 207, 0.3); border-radius: 8px; padding: 25px; margin-bottom: 30px; position: relative; overflow: hidden; box-shadow: 0 4px 20px rgba(0, 0, 0, 0.5);">
<div style="position: absolute; top: 0; left: 0; width: 100%; height: 4px; background: linear-gradient(90deg, #2A6B8C, #5BA4CF, #8BCAE6, #5BA4CF, #2A6B8C); opacity: 0.7;"></div>
<p style="color: #B8D4E8; margin-bottom: 0;">Apache 2.0, inherited down from Gemma.</p>
</div>
<span style="color: #5BA4CF; font-size: 1.8rem; border-bottom: 1px solid rgba(91, 164, 207, 0.3); padding-bottom: 0.3em; display: block;">Merge Details</span>
<div style="background-color: #121A24; border: 1px solid rgba(91, 164, 207, 0.3); border-radius: 8px; padding: 25px; margin-bottom: 30px; position: relative; overflow: hidden; box-shadow: 0 4px 20px rgba(0, 0, 0, 0.5);">
<div style="position: absolute; top: 0; left: 0; width: 100%; height: 4px; background: linear-gradient(90deg, #2A6B8C, #5BA4CF, #8BCAE6, #5BA4CF, #2A6B8C); opacity: 0.7;"></div>
<p style="color: #B8D4E8; margin-bottom: 1em;">This is a merge of pre-trained language models created using <a href="https://github.com/cg123/mergekit" style="color: #5BA4CF; text-decoration: none; border-bottom: 1px dotted rgba(91, 164, 207, 0.4);">mergekit</a>.</p>
<h3 style="color: #D4E8F5; font-size: 1.2rem; margin-top: 0.5em; margin-bottom: 0.5em;">Merge Method</h3>
<p style="color: #B8D4E8; margin-bottom: 1em;">This model was merged using the <a href="https://arxiv.org/abs/2406.11617" style="color: #5BA4CF; text-decoration: none; border-bottom: 1px dotted rgba(91, 164, 207, 0.4);">DELLA</a> merge method using <code style="background-color: rgba(91, 164, 207, 0.1); border-radius: 3px; padding: 0.1em 0.35em; color: #8BCAE6; font-family: 'Courier New', monospace;">gemma-4-31B-it</code> as a base.</p>
<h3 style="color: #D4E8F5; font-size: 1.2rem; margin-top: 1.5em; margin-bottom: 0.5em;">Models Merged</h3>
<p style="color: #B8D4E8; margin-bottom: 0.5em;">The following models were included in the merge:</p>
<ul style="color: #B8D4E8; margin-bottom: 1em; padding-left: 20px;">
<li style="margin-bottom: 0.5em;"><a href="https://huggingface.co/BeaverAI/Artemis-31B-v1h-GGUF" style="color: #5BA4CF; text-decoration: none; border-bottom: 1px dotted rgba(91, 164, 207, 0.4);">BeaverAI/Artemis-31B-v1h-GGUF</a></li>
<li style="margin-bottom: 0.5em;"><a href="https://huggingface.co/zerofata/G4-MeroMero-31B" style="color: #5BA4CF; text-decoration: none; border-bottom: 1px dotted rgba(91, 164, 207, 0.4);">zerofata/G4-MeroMero-31B</a></li>
<li style="margin-bottom: 0.5em;"><a href="https://huggingface.co/llmfan46/gemma-4-Ortenzya-The-Creative-Wordsmith-31B-it-uncensored-heretic" style="color: #5BA4CF; text-decoration: none; border-bottom: 1px dotted rgba(91, 164, 207, 0.4);">llmfan46/gemma-4-Ortenzya-The-Creative-Wordsmith-31B-it-uncensored-heretic</a></li>
</ul>
</div>