Mistral-Medium-3.5-128B-BF16-Text-Only

<style> body { font-family: 'Quicksand', sans-serif; background: linear-gradient(135deg, #4a1e00 0%, #1c0a00 100%); color: #F5EFE6; margin: 0; padding: 0; font-size: 16px; } h1, h2, h3, h4, summary { font-family: 'Cinzel', serif; } .container { margin: 20px auto; max-width: 900px; background-color: rgba(28, 22, 18, 0.95); padding: 30px; border-radius: 12px; box-shadow: 0 4px 20px rgba(255, 140, 0, 0.15); border: 1px solid rgba(255, 140, 0, 0.2); outline: 1px solid rgba(255, 140, 0, 0.5); outline-offset: -1px; position: relative; } .container::before { content: ''; position: absolute; top: -1px; left: -1px; right: -1px; bottom: -1px; border: 1px solid rgba(255, 165, 0, 0.98); border-radius: 12px; pointer-events: none; animation: borderGlow 2.5s ease-in-out infinite; } @keyframes borderGlow { 0% { box-shadow: 0 0 5px rgba(255, 165, 0, 0.98); } 50% { box-shadow: 0 0 12px rgba(255, 165, 0, 0.98); } 100% { box-shadow: 0 0 5px rgba(255, 165, 0, 0.98); } } .header h1 { font-size: 32px; color: #FFA500; margin: 0 0 20px 0; text-align: center; text-shadow: 0 0 12px rgba(255, 100, 0, 0.6); } a { color: #FFD700; text-decoration: none; transition: color 0.3s ease; } .button { display: inline-block; background-color: #E55B00; color: #FFFFFF; padding: 12px 24px; border-radius: 5px; cursor: pointer; text-decoration: none; font-family: 'Cinzel', serif; font-weight: 600; transition: all 0.3s ease; border: 1px solid transparent; } .button:hover { background-color: #FF8C00; box-shadow: 0 0 15px rgba(255, 140, 0, 0.5); transform: translateY(-2px); } .info-card { background: rgba(45, 35, 25, 0.95); border: 1px solid rgba(255, 140, 0, 0.2); border-radius: 8px; overflow: hidden; margin-bottom: 25px; } .info-header { background: rgba(255, 140, 0, 0.1); padding: 20px; border-bottom: 1px solid rgba(255, 140, 0, 0.2); } .info-header h3 { color: #FFA500; margin: 0 0 10px 0; font-size: 22px; } .card-content { padding: 20px; line-height: 1.7; } .card-content ul { list-style: none; padding-left: 20px; } .card-content li::before { content: '✦'; color: #FFD700; font-weight: bold; display: inline-block; width: 1em; margin-left: -1.2em; } .card-content strong { color: #FFD700; } /* Update to the note card */ .note-card { border: 1px solid #FFA500; box-shadow: 0 0 10px rgba(255, 165, 0, 0.1); } .note-header { background: rgba(255, 165, 0, 0.1); } .note-header h3 { color: #FFA500; text-align: center; } .support-section { text-align: center; margin-top: 40px; background: rgba(45, 35, 25, 0.95); border: 1px solid rgba(255, 140, 0, 0.2); border-radius: 8px; padding: 20px; } summary { cursor: pointer; list-style: none; outline: none; display: flex; align-items: center; } summary::before { content: '▶'; font-size: 1.2em; color: #FFA500; margin-right: 15px; transition: transform 0.2s ease; } details[open] > summary::before { transform: rotate(90deg); } h2 { color: #FFA500; border-bottom: 1px solid rgba(255, 140, 0, 0.2); padding-bottom: 10px; margin-bottom: 20px; } </style> <div class="container"> <link href="https://fonts.googleapis.com/css2?family=Cinzel:wght@400;500;600&family=Quicksand:wght@400;500&display=swap" rel="stylesheet"> <div class="header"> <h1>Mistral-Medium-3.5-128B-BF16-Text-Only</h1> </div> <div class="info"> <div class="info-card note-card"> <div class="info-header note-header"> <h3>📜 Technical Architecture Note</h3> </div> <div class="card-content"> <p>This model has been converted from <strong>Mistral3ForConditionalGeneration</strong> (Multimodal) to <strong>MistralForCausalLM</strong> (Standard Text-Only). This change ensures maximum compatibility with standard fine-tuning libraries like <em>Axolotl, Unsloth, and Hugging Face Transformers</em> without requiring custom vision-encoder handling.</p> </div> </div> <div class="support-section"> <p><strong>Help me feed the data beast! Taking commissions for universe-specific models.</strong></p> <a href="https://ko-fi.com/som1tokmynam" target="_blank" class="button"> Support on Ko-fi </a> </div> <div class="section-container"> <details open> <summary><h2>Model Description</h2></summary> <div class="info-card"> <div class="card-content"> <p>This is a processed version of <strong>Mistral-Medium-3.5-128B</strong> designed for users who prioritize text-only performance and ease of fine-tuning.</p> <p><strong>Modification Details:</strong></p> <ul> <li><strong>Precision Upscale:</strong> Converted from <strong>FP8</strong> weights to <strong>BF16</strong> to restore full 16-bit brain-float precision for stable gradient updates during training.</li> <li><strong>Vision Layer Stripping:</strong> All vision encoders and multimodal projection layers have been removed, significantly reducing memory overhead during inference and training for text-only tasks.</li> <li><strong>Architecture Re-mapping:</strong> The configuration has been modified to use <code>MistralForCausalLM</code>, allowing it to be treated as a standard dense language model.</li> </ul> </div> </div> </details> </div> <div class="section-container"> <details> <summary><h2>Purpose & Usage</h2></summary> <div class="info-card"> <div class="card-content"> <p>This model is intended to serve as a <strong>clean base for fine-tuning</strong>. By removing the vision components, you can allocate more VRAM to sequence length or batch size. It is 100% functional for text-only chat and reasoning out of the box.</p> </div> </div> </details> </div> <div class="section-container"> <details> <summary><h2>Acknowledgements</h2></summary> <div class="info-card"> <div class="card-content"> <ul> <li>Credit to <strong>Mistral AI</strong> for the original Mistral-Medium-3.5-128B architecture.</li> </ul> </div> </div> </details> </div> </div> </div>