Downloads · 30 days
194
1% of all-time downloads
GLM-5.3-GGUF
L-Alchemyst/GLM-5.3-GGUF
GLM-5.3-GGUF is a machine learning model from L-Alchemyst. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as other.
<style .qz-wrap { background: 0d1117; color: e6edf3; font-family: 'Inter', system-ui, -apple-system, sans-serif; font-size: 1rem; line-height: 1.7; padding: 2rem 2.5rem; border-radius: 8px; margin-bottom: 2rem; } .qz-…
All-time downloads
22.2K
Public
Repo size
1 TB
Likes
8
Public
Repo makeup
Click a slice to open those files.
.gguf1 TB · 100%
At a glance
- License
- other
- Access
- Public
- Created
- Aug 29, 2026
- Updated
- Sep 7, 2026
- SHA
- 9e2812c1
Base models
Model card
From the Hugging Face model README
<style>
.qz-wrap {
background: #0d1117;
color: #e6edf3;
font-family: 'Inter', system-ui, -apple-system, sans-serif;
font-size: 1rem;
line-height: 1.7;
padding: 2rem 2.5rem;
border-radius: 8px;
margin-bottom: 2rem;
}
.qz-wrap * { color: #e6edf3; }
.qz-wrap a { color: #58a6ff; text-decoration: none; }
.qz-wrap a:hover { text-decoration: underline; }
.qz-header {
font-size: 2.2rem;
font-weight: 700;
color: #ffffff;
border-bottom: 3px solid #2e7d5e;
padding-bottom: 0.5rem;
margin-bottom: 1rem;
}
.qz-subtitle {
font-size: 1.05rem;
color: #8b949e;
margin-top: 0.2rem;
margin-bottom: 0.4rem;
}
.qz-section {
font-size: 1.3rem;
font-weight: 600;
color: #ffffff;
margin-top: 2rem;
margin-bottom: 0.6rem;
}
.qz-sub {
font-size: 1.1rem;
font-weight: 600;
color: #7ab8d4;
margin-top: 1.4rem;
margin-bottom: 0.5rem;
}
.qz-wrap table.qz-table {
display: table;
width: 100%;
min-width: 100%;
max-width: 100%;
table-layout: fixed;
border-collapse: collapse;
border-spacing: 0;
margin-bottom: 1rem;
}
.qz-wrap table.qz-table th,
.qz-wrap table.qz-table td {
padding: 0.55rem 1rem;
text-align: left;
border-bottom: 1px solid #30363d;
box-sizing: border-box;
}
.qz-wrap table.qz-table th:first-child,
.qz-wrap table.qz-table td:first-child {
width: 30%;
}
.qz-wrap table.qz-table th:last-child,
.qz-wrap table.qz-table td:last-child {
width: 70%;
}
.qz-wrap table.qz-table th {
background: rgba(46, 125, 94, 0.08);
color: #5eb88a;
font-weight: 700;
text-transform: uppercase;
font-size: 0.72rem;
letter-spacing: 0.06em;
}
.qz-wrap table.qz-table tr:nth-child(even) td {
background: rgba(255, 255, 255, 0.02);
}
.qz-note {
font-size: 0.88rem;
color: #8b949e;
font-style: italic;
}
.qz-status {
font-size: 0.9rem;
color: #d69e2e;
font-weight: 500;
padding: 0.6rem 1rem;
border: 1px solid #d69e2e;
border-radius: 4px;
display: inline-block;
margin-bottom: 1rem;
}
.qz-recipe-toggle {
margin: 0.8rem 0 1rem 0;
border: 1px solid #0891b2;
border-radius: 6px;
overflow: hidden;
width: 100%;
}
.qz-recipe-toggle summary {
font-size: 0.9rem;
font-weight: 600;
color: #22d3ee;
padding: 0.6rem 1rem;
cursor: pointer;
user-select: none;
background: rgba(34, 211, 238, 0.06);
list-style: none;
}
.qz-recipe-toggle summary::before { content: '▸ '; }
.qz-recipe-toggle[open] summary::before { content: '▾ '; }
.qz-recipe-toggle summary:hover { background: rgba(34, 211, 238, 0.12); }
.qz-recipe-body {
font-family: 'JetBrains Mono', 'Fira Code', monospace;
font-size: 0.8rem;
padding: 0.8rem 1rem;
line-height: 1.7;
background: rgba(34, 211, 238, 0.03);
color: #e6edf3;
}
.qz-recipe-body .rq-section {
color: #22d3ee;
font-weight: 700;
margin-top: 0.6rem;
}
.qz-recipe-body .rq-line { display: block; }
.qz-recipe-body .rq-quant {
color: #67e8f9;
font-weight: 600;
}
.qz-code {
font-family: 'JetBrains Mono', 'Fira Code', monospace;
font-size: 0.82rem;
display: block;
padding: 1rem;
margin: 0.8rem 0;
overflow-x: auto;
border-left: 3px solid #0891b2;
background: rgba(34, 211, 238, 0.04);
color: #e6edf3;
line-height: 1.7;
white-space: pre;
}
.qz-chart-container {
text-align: center;
margin: 0.5rem 0 1.5rem 0;
}
.qz-chart-container img {
border: 1px solid #30363d;
border-radius: 4px;
max-width: 100%;
height: auto;
}
.qz-credit {
font-size: 0.9rem;
color: #8b949e;
margin-top: 2rem;
padding-top: 1rem;
border-top: 1px solid #30363d;
}
.qz-credit a { color: #5eb88a; }
</style>
<html>
<head>
<meta charset="utf-8">
</head>
<body>
<div class="qz-wrap">
<div class="qz-header">GLM-5.3</div>
<div class="qz-subtitle">Imatrix quants best-in-class for <a href="https://github.com/ikawrakow/ik_llama.cpp">ik_llama.cpp</a></div>
<div class="qz-subtitle">NOTE: ik_llama.cpp is highly versatile and runs mainline llama.cpp quants such as existing GGUFs from bartowski, unsloth, mradermacher, Aes Sedai etc</div>
<div class="qz-status">IQ2_K done! ✅ IQ3_K done! ✅ IQ4_K done! ✅. All quants done unless someone really wants a Q1. See the collection below.</div>
<div class="qz-section">General Specs</div>
<table class="qz-table">
<tr><th></th><th>Value</th></tr>
<tr><td>Architecture</td><td>GLM-DSA (MoE, 256 experts × 22B, 8 active)</td></tr>
<tr><td>Params</td><td>744B total / 40B active</td></tr>
<tr><td>Context</td><td>1,048,576</td></tr>
<tr><td>Base</td><td><a href="https://huggingface.co/zai-org/GLM-5.3-BF16">zai-org/GLM-5.3-BF16</a></td></tr>
<tr><td>Imatrix</td><td><a href="https://huggingface.co/AesSedai">Aes Sedai</a> — converted for ik_llama.cpp</td></tr>
<tr><td>Calibration</td><td>calibration_datav5.txt</td></tr>
</table>
<div class="qz-section">PPL Analysis</div>
<div class="qz-chart-container">
<img src="ppl_vs_size.png" alt="Relative PPL vs Model Size">
</div>
<div class="qz-section">Quant Collection</div>
<!-- IQ4_K -->
<div class="qz-sub">✅ IQ4_K <span class="qz-note">4.581 BPW · 401.77 GiB</span></div>
<div class="qz-note">Evaluated on wiki.test.raw, 512 ctx, 565 chunks, against BF16 reference logits.</div>
<details class="qz-recipe-toggle">
<summary>Why the specific layer targeting? (Analysis notes)</summary>
<div class="qz-recipe-body">
Rather than relying solely on conventional depth-based heuristics, the quantization targets were selected using <b>imatrix activation-energy analysis</b> and residuals after detrending for the model's natural increase in activation energy with depth.
The analysis identified <b>blk.3–11 and blk.76–77</b> as unusually low-energy relative to their expected depth, making them candidates for more aggressive <code>iq4_ks</code> quantization. This is intentionally non-conventional. Since the measured sensitivity profile for these layers was weaker than expected.
In contrast, the mid-depth MoE layers, particularly around <b>blk.29–40</b>, show substantially higher residual energy and were retained at <code>iq4_k</code>. The <b>expert down projections</b> were also treated more conservatively because they feed directly back into the residual stream.
<b>Detrended layer ranking (down output energy):</b><br> Trend slope: d(log E)/d(layer) = 0.1545. Residual = actual/trend; lower residual ⇒ weaker than depth predicts.
<div class="qz-code"> blk.3 resid=0.016 up_flow=2.331e+06 blk.4 resid=0.075 up_flow=3.914e+06 blk.5 resid=0.110 up_flow=4.457e+06 blk.6 resid=0.209 up_flow=7.723e+06 blk.7 resid=0.289 up_flow=1.349e+07 blk.76 resid=0.292 up_flow=5.089e+09 blk.10 resid=0.325 up_flow=2.200e+07 blk.9 resid=0.349 up_flow=2.073e+07 blk.77 resid=0.350 up_flow=2.786e+09 blk.11 resid=0.355 up_flow=2.457e+07 blk.75 resid=0.364 up_flow=6.132e+09 blk.67 resid=0.377 up_flow=4.028e+09 ... blk.36 resid=4.166 up_flow=9.199e+08 blk.32 resid=4.225 up_flow=6.227e+08 blk.34 resid=4.276 up_flow=7.851e+08 blk.33 resid=4.401 up_flow=7.199e+08 blk.35 resid=4.430 up_flow=8.652e+08</div> <br> The resulting recipe is therefore <b>sensitivity-guided rather than position-guided</b>: quantize the measured low-sensitivity outliers slightly more, while preserving precision through the high-energy middle of the network. </div> </details> <table class="qz-table"> <tr><th>Metric</th><th>Value</th></tr> <tr><td>PPL (quant)</td><td>2.707673 ± 0.014047</td></tr> <tr><td>PPL (BF16 ref)</td><td>2.674315 ± 0.013811</td></tr> <tr><td>(PPL(Q)/PPL(base)) - 1</td><td>+1.2473% ± 0.0963%</td></tr> <tr><td>KLD</td><td>0.041196 ± 0.000334</td></tr> <tr><td>Same top-p</td><td>93.941 ± 0.063 %</td></tr> <tr><td>Δp RMS</td><td>7.839 ± 0.052 %</td></tr> </table> <details class="qz-recipe-toggle"> <summary>Quantization recipe</summary> <div class="qz-recipe-body"> <span class="rq-section"># Attention</span> <span class="rq-line">blk\..*\.attn_k_b\.weight=<span class="rq-quant">q8_0</span></span> <span class="rq-line">blk\..*\.attn_v_b\.weight=<span class="rq-quant">q8_0</span></span> <span class="rq-line">blk\..*\.attn_kv_a_mqa\.weight=<span class="rq-quant">q8_0</span></span> <span class="rq-line">blk\..*\.attn_q_a\.weight=<span class="rq-quant">q8_0</span></span> <span class="rq-line">blk\..*\.attn_q_b\.weight=<span class="rq-quant">q8_0</span></span> <span class="rq-line">blk\..*\.attn_output\.weight=<span class="rq-quant">q8_0</span></span><span class="rq-section"># Router</span> <span class="rq-line">blk\..*\.ffn_gate_inp\.weight=<span class="rq-quant">f32</span></span>
<span class="rq-section"># First 3 Dense Layers</span> <span class="rq-line">blk\..*\.ffn_down\.weight=<span class="rq-quant">iq6_k</span></span> <span class="rq-line">blk\..*\.ffn_(gate|up)\.weight=<span class="rq-quant">iq5_ks</span></span>
<span class="rq-section"># Shared Expert Layers</span> <span class="rq-line">blk\..*\.ffn_down_shexp\.weight=<span class="rq-quant">iq6_k</span></span> <span class="rq-line">blk\..*\.ffn_(gate|up)_shexp\.weight=<span class="rq-quant">iq6_k</span></span>
<span class="rq-section"># Routed Experts</span> <span class="rq-line">blk\.(78)\.ffn_down_exps\.weight=<span class="rq-quant">q6_K</span></span> <span class="rq-line">blk\.(78)\.ffn_(gate|up)_exps\.weight=<span class="rq-quant">q6_K</span></span> <span class="rq-line">blk\.(3|4|5|6|7|8|9|10|11|76|77)\.ffn_down_exps\.weight=<span class="rq-quant">iq4_ks</span></span> <span class="rq-line">blk\.(3|4|5|6|7|8|9|10|11|76|77)\.ffn_(gate|up)_exps\.weight=<span class="rq-quant">iq4_ks</span></span> <span class="rq-line">blk\..*\.ffn_down_exps\.weight=<span class="rq-quant">iq4_k</span></span> <span class="rq-line">blk\..*\.ffn_(gate|up)_exps\.weight=<span class="rq-quant">iq4_k</span></span>
<span class="rq-section"># Indexer</span> <span class="rq-line">blk\..*\.indexer\.attn_q_b\.weight=<span class="rq-quant">q6_K</span></span> <span class="rq-line">blk\..*\.indexer\.attn_k\.weight=<span class="rq-quant">q8_0</span></span> <span class="rq-line">blk\..*\.indexer\.proj\.weight=<span class="rq-quant">q8_0</span></span> <span class="rq-line">blk\..*\.indexer\.k_norm\.(weight|bias)=<span class="rq-quant">f32</span></span>
<span class="rq-section"># MTP block - eh_proj is the throughput-critical tensor per ik's finding</span> <span class="rq-line">blk\..*\.nextn\.eh_proj\.weight=<span class="rq-quant">q4_0</span></span> <span class="rq-line">blk\..*\.nextn\.enorm\.weight=<span class="rq-quant">f32</span></span> <span class="rq-line">blk\..*\.nextn\.hnorm\.weight=<span class="rq-quant">f32</span></span> <span class="rq-line">blk\..*\.nextn\.shared_head_norm\.weight=<span class="rq-quant">f32</span></span>
<span class="rq-line">token_embd\.weight=<span class="rq-quant">iq6_k</span></span> <span class="rq-line">output\.weight=<span class="rq-quant">iq6_k</span></span>
</div> </details> <!-- IQ3_K --> <div class="qz-sub">✅ IQ3_K <span class="qz-note">3.670 BPW · 321.83 GiB</span></div> <div class="qz-note">Evaluated on wiki.test.raw, 512 ctx, 565 chunks, against BF16 reference logits.</div> <table class="qz-table"> <tr><th>Metric</th><th>Value</th></tr> <tr><td>PPL (quant)</td><td>2.820603 ± 0.014842</td></tr> <tr><td>PPL (BF16 ref)</td><td>2.674315 ± 0.013811</td></tr> <tr><td>(PPL(Q)/PPL(base)) - 1</td><td>+5.4701% ± 0.1627%</td></tr> <tr><td>KLD</td><td>0.108800 ± 0.000809</td></tr> <tr><td>Same top-p</td><td>90.352 ± 0.078 %</td></tr> <tr><td>Δp RMS</td><td>12.780 ± 0.070 %</td></tr> </table> <details class="qz-recipe-toggle"> <summary>Quantization recipe</summary> <div class="qz-recipe-body"> <span class="rq-section"># Attention</span> <span class="rq-line">blk\..*\.attn_k_b\.weight=<span class="rq-quant">q8_0</span></span> <span class="rq-line">blk\..*\.attn_v_b\.weight=<span class="rq-quant">q8_0</span></span> <span class="rq-line">blk\..*\.attn_kv_a_mqa\.weight=<span class="rq-quant">q8_0</span></span> <span class="rq-line">blk\..*\.attn_q_a\.weight=<span class="rq-quant">iq6_k</span></span> <span class="rq-line">blk\..*\.attn_q_b\.weight=<span class="rq-quant">iq6_k</span></span> <span class="rq-line">blk\..*\.attn_output\.weight=<span class="rq-quant">iq6_k</span></span><span class="rq-section"># Router</span> <span class="rq-line">blk\..*\.ffn_gate_inp\.weight=<span class="rq-quant">f32</span></span>
<span class="rq-section"># First 3 Dense Layers</span> <span class="rq-line">blk\..*\.ffn_down\.weight=<span class="rq-quant">iq6_k</span></span> <span class="rq-line">blk\..*\.ffn_(gate|up)\.weight=<span class="rq-quant">iq5_ks</span></span>
<span class="rq-section"># Shared Expert Layers</span> <span class="rq-line">blk\..*\.ffn_down_shexp\.weight=<span class="rq-quant">iq6_k</span></span> <span class="rq-line">blk\..*\.ffn_(gate|up)_shexp\.weight=<span class="rq-quant">iq6_k</span></span>
<span class="rq-section"># Routed Experts</span> <span class="rq-line">blk\.(78)\.ffn_down_exps\.weight=<span class="rq-quant">q6_K</span></span> <span class="rq-line">blk\.(78)\.ffn_(gate|up)_exps\.weight=<span class="rq-quant">q6_K</span></span> <span class="rq-line">blk\..*\.ffn_down_exps\.weight=<span class="rq-quant">iq4_ks</span></span> <span class="rq-line">blk\..*\.ffn_(gate|up)_exps\.weight=<span class="rq-quant">iq3_ks</span></span>
<span class="rq-section"># Indexer</span> <span class="rq-line">blk\..*\.indexer\.k_norm\.(weight|bias)=<span class="rq-quant">f32</span></span> <span class="rq-line">blk\..*\.indexer\.proj\.weight=<span class="rq-quant">q8_0</span></span> <span class="rq-line">blk\..*\.indexer\.attn_k\.weight=<span class="rq-quant">q8_0</span></span> <span class="rq-line">blk\..*\.indexer\.attn_q_b\.weight=<span class="rq-quant">q6_K</span></span>
<span class="rq-section"># MTP block - eh_proj is the throughput-critical tensor per ik's finding</span> <span class="rq-line">blk\..*\.nextn\.eh_proj\.weight=<span class="rq-quant">q4_0</span></span> <span class="rq-line">blk\..*\.nextn\.enorm\.weight=<span class="rq-quant">f32</span></span> <span class="rq-line">blk\..*\.nextn\.hnorm\.weight=<span class="rq-quant">f32</span></span> <span class="rq-line">blk\..*\.nextn\.shared_head_norm\.weight=<span class="rq-quant">f32</span></span>
<span class="rq-line">token_embd\.weight=<span class="rq-quant">iq5_k</span></span> <span class="rq-line">output\.weight=<span class="rq-quant">iq6_k</span></span>
</div> </details> <!-- IQ2_K --> <div class="qz-sub">✅ IQ2_K <span class="qz-note">2.695 BPW · 236.39 GiB</span></div> <div class="qz-note"><strong>The Q2 to run on 256 GiB systems.</strong> Tight 256 GiB-friendly footprint at 2.695 BPW. GGUF upload in progress — see status above.</div> <div class="qz-note">Evaluated on wiki.test.raw, 512 ctx, 565 chunks, against BF16 reference logits.</div> <table class="qz-table"> <tr><th>Metric</th><th>Value</th></tr> <tr><td>PPL (quant)</td><td>3.436789 ± 0.018731</td></tr> <tr><td>PPL (BF16 ref)</td><td>2.674315 ± 0.013811</td></tr> <tr><td>(PPL(Q)/PPL(base)) - 1</td><td>+28.511% ± 0.3730%</td></tr> <tr><td>KLD</td><td>0.381602 ± 0.002102</td></tr> <tr><td>Same top-p</td><td>80.498 ± 0.104 %</td></tr> <tr><td>Δp RMS</td><td>24.399 ± 0.087 %</td></tr> </table> <details class="qz-recipe-toggle"> <summary>Quantization recipe</summary> <div class="qz-recipe-body"> <span class="rq-section"># Attention</span> <span class="rq-line">blk\..*\.attn_k_b\.weight=<span class="rq-quant">q8_0</span></span> <span class="rq-line">blk\..*\.attn_v_b\.weight=<span class="rq-quant">q8_0</span></span> <span class="rq-line">blk\..*\.attn_kv_a_mqa\.weight=<span class="rq-quant">q8_0</span></span> <span class="rq-line">blk\..*\.attn_q_a\.weight=<span class="rq-quant">q8_0</span></span> <span class="rq-line">blk\..*\.attn_q_b\.weight=<span class="rq-quant">q8_0</span></span> <span class="rq-line">blk\..*\.attn_output\.weight=<span class="rq-quant">iq6_k</span></span><span class="rq-section"># Router</span> <span class="rq-line">blk\..*\.ffn_gate_inp\.weight=<span class="rq-quant">f32</span></span>
<span class="rq-section"># First 3 Dense Layers</span> <span class="rq-line">blk\..*\.ffn_down\.weight=<span class="rq-quant">iq6_k</span></span> <span class="rq-line">blk\..*\.ffn_(gate|up)\.weight=<span class="rq-quant">iq5_ks</span></span>
<span class="rq-section"># Shared Expert Layers</span> <span class="rq-line">blk\..*\.ffn_down_shexp\.weight=<span class="rq-quant">iq6_k</span></span> <span class="rq-line">blk\..*\.ffn_(gate|up)_shexp\.weight=<span class="rq-quant">iq6_k</span></span>
<span class="rq-section"># Routed Experts</span> <span class="rq-line">blk\.(78)\.ffn_down_exps\.weight=<span class="rq-quant">q6_K</span></span> <span class="rq-line">blk\.(78)\.ffn_(gate|up)_exps\.weight=<span class="rq-quant">q6_K</span></span> <span class="rq-line">blk\..*\.ffn_down_exps\.weight=<span class="rq-quant">iq3_ks</span></span> <span class="rq-line">blk\..*\.ffn_(gate|up)_exps\.weight=<span class="rq-quant">iq2_ks</span></span>
<span class="rq-section"># Indexer</span> <span class="rq-line">blk\..*\.indexer\.attn_q_b\.weight=<span class="rq-quant">q6_K</span></span> <span class="rq-line">blk\..*\.indexer\.attn_k\.weight=<span class="rq-quant">q8_0</span></span> <span class="rq-line">blk\..*\.indexer\.proj\.weight=<span class="rq-quant">q8_0</span></span> <span class="rq-line">blk\..*\.indexer\.k_norm\.(weight|bias)=<span class="rq-quant">f32</span></span>
<span class="rq-section"># MTP block - eh_proj is the throughput-critical tensor per ik's finding</span> <span class="rq-line">blk\..*\.nextn\.eh_proj\.weight=<span class="rq-quant">q4_0</span></span> <span class="rq-line">blk\..*\.nextn\.enorm\.weight=<span class="rq-quant">f32</span></span> <span class="rq-line">blk\..*\.nextn\.hnorm\.weight=<span class="rq-quant">f32</span></span> <span class="rq-line">blk\..*\.nextn\.shared_head_norm\.weight=<span class="rq-quant">f32</span></span>
<span class="rq-line">token_embd\.weight=<span class="rq-quant">iq6_k</span></span> <span class="rq-line">output\.weight=<span class="rq-quant">iq6_k</span></span>
</div> </details> <div class="qz-section">Quick Start</div> <div class="qz-note">Requires <a href="https://github.com/ikawrakow/ik_llama.cpp">ik_llama.cpp</a>. Standard llama.cpp will not run these quants.</div><span class="qz-code"># Clone and build git clone https://github.com/ikawrakow/ik_llama.cpp cd ik_llama.cpp cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON cmake --build build --config Release -j $(nproc)</span>
<span class="qz-code"># Download pip install huggingface_hub hf download L-Alchemyst/GLM-5.3-GGUF --repo-type model --include "IQ3_K/*" --local-dir ./glm5.3-iq3k</span>
<span class="qz-code"># Hybrid CPU+GPU — adjust --n-cpu-moe and -cram to your RAM
./build/bin/llama-server
--model GLM-5.3-IQ3_K.gguf
-muge --merge-qkv
--ctx-size 131072
-ctk f16 -mla 3 -amb 1536
-ngl 999
--n-cpu-moe 50
--parallel 1
--threads 96 --threads-batch 128
--host 127.0.0.1 --port 8080
--no-mmap
-cram 8192
--jinja</span>
- License
- other
- Created
- Aug 29, 2026
- Updated
- Sep 7, 2026