Xenon-26B-A4B-OPAL-GGUF

Note: this repo also contains a tool call grammar file (tool_call.gbnf), thats meant to be loaded in llama.cpp, IT IS REQUIRED FOR AGENTIC TASKS

<div style="font-family: -apple-system, BlinkMacSystemFont, 'Geist', Roboto, sans-serif; border: 1px solid #155e75; border-radius: 18px; box-shadow: 0 18px 40px rgba(6, 182, 212, 0.14); overflow: hidden; background: #020617; margin-bottom: 30px;"> <div style="background: radial-gradient(circle at 15% 15%, rgba(16, 185, 129, 0.25) 0%, rgba(6, 182, 212, 0.12) 28%, transparent 46%), linear-gradient(135deg, #042f2e 0%, #0c4a6e 58%, #0e7490 100%); padding: 28px; color: white;"> <div style="display: flex; align-items: center; justify-content: space-between; flex-wrap: wrap; gap: 12px;"> <h1 style="margin: 0; font-size: 28px; font-weight: 900; display: flex; align-items: center; gap: 12px; color: #ecfeff; border: none;"><a href="https://huggingface.co/spaces/el4/OPAL" display: flex; align-items: center; gap: 12px;">💎 OPAL</a></h1> <span style="background: #06b6d4; color: #042f2e; font-size: 11px; font-weight: 900; padding: 5px 12px; border-radius: 999px; text-transform: uppercase; letter-spacing: 0.7px; box-shadow: 0 0 12px rgba(6, 182, 212, 0.5);">Consumer-GPU Quantization Harness</span> </div> <p style="margin: 10px 0 0 0; font-size: 14px; color: #a5f3fc; font-weight: 600;">This repo contains OPAL Quants of el4/Xenon-26B-A4B</p> </div> <div style="padding: 24px;"> <table style="width: 100%; border-collapse: collapse; font-family: inherit; font-size: 13px;"> <thead> <tr style="background: rgba(20, 184, 166, 0.15);"> <th style="padding: 10px; border-bottom: 2px solid #14b8a6; text-align: left; color: #5eead4; font-weight: 900;">Tier Name</th> <th style="padding: 10px; border-bottom: 2px solid #14b8a6; text-align: left; color: #5eead4; font-weight: 900;">Target Size</th> <th style="padding: 10px; border-bottom: 2px solid #14b8a6; text-align: left; color: #5eead4; font-weight: 900;">Middle Layer Strategy</th> </tr> </thead> <tbody> <tr><td style="padding: 10px; border-bottom: 1px solid rgba(20, 184, 166, 0.2); font-weight: 800; color: #e2e8f0;">💎 <code style="color:#67e8f9;">quality</code></td><td style="padding: 10px; border-bottom: 1px solid rgba(20, 184, 166, 0.2); color: #94a3b8;">~23 GB</td><td style="padding: 10px; border-bottom: 1px solid rgba(20, 184, 166, 0.2); color: #94a3b8;">IQ4_XS</td></tr> <tr style="background: rgba(12, 74, 110, 0.2);"><td style="padding: 10px; border-bottom: 1px solid rgba(20, 184, 166, 0.2); font-weight: 800; color: #e2e8f0;">⚖️ <code style="color:#67e8f9;">balanced</code></td><td style="padding: 10px; border-bottom: 1px solid rgba(20, 184, 166, 0.2); color: #94a3b8;">~25 GB</td><td style="padding: 10px; border-bottom: 1px solid rgba(20, 184, 166, 0.2); color: #94a3b8;">Q5_K</td></tr> <tr><td style="padding: 10px; border-bottom: 1px solid rgba(20, 184, 166, 0.2); font-weight: 800; color: #e2e8f0;">📦 <code style="color:#67e8f9;">compact</code></td><td style="padding: 10px; border-bottom: 1px solid rgba(20, 184, 166, 0.2); color: #94a3b8;">~18 GB</td><td style="padding: 10px; border-bottom: 1px solid rgba(20, 184, 166, 0.2); color: #94a3b8;">Q3_K</td></tr> <tr style="background: rgba(12, 74, 110, 0.2);"><td style="padding: 10px; border-bottom: 1px solid rgba(20, 184, 166, 0.2); font-weight: 800; color: #e2e8f0;">🚀 <code style="color:#67e8f9;">mini</code></td><td style="padding: 10px; border-bottom: 1px solid rgba(20, 184, 166, 0.2); color: #94a3b8;">~14 GB</td><td style="padding: 10px; border-bottom: 1px solid rgba(20, 184, 166, 0.2); color: #94a3b8;">IQ2_S</td></tr> <tr><td style="padding: 10px; border-bottom: 1px solid rgba(20, 184, 166, 0.2); font-weight: 800; color: #e2e8f0;">🔬 <code style="color:#67e8f9;">micro</code></td><td style="padding: 10px; border-bottom: 1px solid rgba(20, 184, 166, 0.2); color: #94a3b8;">~11 GB</td><td style="padding: 10px; border-bottom: 1px solid rgba(20, 184, 166, 0.2); color: #94a3b8;">IQ1_M</td></tr> <tr style="background: rgba(12, 74, 110, 0.2);"><td style="padding: 10px; border-bottom: 1px solid rgba(20, 184, 166, 0.2); font-weight: 800; color: #e2e8f0;">🤏 <code style="color:#67e8f9;">nano</code></td><td style="padding: 10px; border-bottom: 1px solid rgba(20, 184, 166, 0.2); color: #94a3b8;">~12 GB</td><td style="padding: 10px; border-bottom: 1px solid rgba(20, 184, 166, 0.2); color: #94a3b8;">IQ2_XXS</td></tr> <tr><td style="padding: 10px; font-weight: 800; color: #e2e8f0;">🛡️ <code style="color:#67e8f9;">exp-minimalist</code></td><td style="padding: 10px; color: #94a3b8;">~16 GB</td><td style="padding: 10px; color: #94a3b8;">IQ2_XXS (Protected Edges)</td></tr> </tbody> </table> </div> </div> <div style="font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif; border: 1px solid #155e75; border-radius: 16px; overflow: hidden; background: #020617; box-shadow: 0 10px 24px rgba(6, 182, 212, 0.10); margin-bottom: 30px;"> <div style="background: linear-gradient(135deg, #042f2e 0%, #0c4a6e 65%, #0e7490 100%); padding: 18px 20px; color: white;"> <h3 style="margin: 0; color: #ecfeff; border: none; font-size: 19px; font-weight: 900;">📚 Credits</h3> </div> <div style="padding: 20px; font-size: 13px; color: #94a3b8; line-height: 1.7;"> <p style="margin: 0 0 12px 0;">👉 <b style="color: #fff;">APEX Quantization Method</b><br>Ettore Di Giacinto & Richard Palethorpe (LocalAI Team). OPAL adapts the layer-wise precision gradients and MoE-aware tensor classification outlined in the APEX technical paper.</p> <p style="margin: 0 0 12px 0;">👉 <b style="color: #fff;">Bartowski and Lamim</b><br>For the excellent semantic imatrix calibration dataset that powers OPAL's activation scaling.</p> <p style="margin: 0 0 12px 0;">👉 <b style="color: #fff;">llama.cpp</b><br>Georgi Gerganov and contributors for the foundational inference and quantization engine.</p> <p style="margin: 0 0 12px 0;">👉 <b style="color: #fff;">HuggingFace Accelerate</b><br>For the <code style="color:#a5f3fc;">init_empty_weights()</code> context manager that makes the 0-RAM "Ghost Model" possible.</p> <p style="margin: 0;">👉 <b style="color: #fff;">Jackrong</b><br>For the sexy markdown readme inspo.</p> </div> </div> <div style="font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif; border: 1px solid #ca8a04; border-radius: 18px; box-shadow: 0 18px 40px rgba(245, 158, 11, 0.12); overflow: hidden; background: #020617; margin-top: 30px; margin-bottom: 30px;"> <div style="background: radial-gradient(circle at 15% 15%, rgba(254, 240, 138, 0.2) 0%, rgba(245, 158, 11, 0.12) 28%, transparent 46%), linear-gradient(135deg, #042f2e 0%, #1c1917 58%, #ca8a04 100%); padding: 20px; color: white;"> <h3 style="margin: 0; color: #fef3c7; border: none; font-size: 18px; font-weight: 900; display: flex; align-items: center; gap: 10px;"> <svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="#fbbf24"><path d="M12 1.75L5.75 12.25L12 16L18.25 12.25L12 1.75M5.75 13.5L12 17.25L18.25 13.5L12 22.25L5.75 13.5Z"/></svg> Support the Project </h3> </div> <div style="padding: 24px; text-align: center; background: #070E12;"> <p style="margin: 0 0 16px 0; font-size: 14px; color: #d1d5db; line-height: 1.6;"> A coffee in Ethereum would be cool! Although I don't drink coffee—I think it tastes like burnt water—but a pink lemonade would be fire! 🔥 </p> <a href="https://etherscan.io/address/0xDEE7fa8C421BD038D32e4441ea1aDe72fE973706" target="_blank" style="display: inline-block; background: #050B0E; border: 1px solid #4b5563; border-radius: 12px; padding: 12px 24px; font-family: 'JetBrains Mono', monospace; font-size: 13px; color: #fbbf24; letter-spacing: 0.5px; text-decoration: underline; text-underline-offset: 4px; transition: all 0.3s ease;"> 0xDEE7fa8C421BD038D32e4441ea1aDe72fE973706 </a> </div> </div>