Ling-3.0-flash-abliterated-APEX-GGUF

<div style="font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif; margin-bottom: 24px;"> <div style="background: #f5f5f7; border-radius: 20px; padding: 36px 32px; margin-bottom: 20px; text-align: center; position: relative; overflow: hidden;"> <div style="position: absolute; top: -30px; right: -30px; width: 120px; height: 120px; background: #a5f3fc; border-radius: 50%;"></div> <div style="position: absolute; bottom: -20px; left: 40px; width: 80px; height: 80px; background: #67e8f9; border-radius: 50%;"></div> <div style="position: absolute; top: 50%; left: -15px; width: 60px; height: 60px; background: #a5f3fc; border-radius: 50%;"></div> <div style="display: inline-flex; gap: 8px; margin-bottom: 16px; position: relative; z-index: 1; flex-wrap: wrap; justify-content: center;"> <span style="background: #0891b2; color: white; font-size: 11px; font-weight: 600; padding: 5px 14px; border-radius: 20px;">ABLITERATED</span> <span style="background: #6366f1; color: white; font-size: 11px; font-weight: 600; padding: 5px 14px; border-radius: 20px;">V6 T21</span> <span style="background: #0ea5e9; color: white; font-size: 11px; font-weight: 600; padding: 5px 14px; border-radius: 20px;">APEX</span> <span style="background: #64748b; color: white; font-size: 11px; font-weight: 600; padding: 5px 14px; border-radius: 20px;">MIT</span> </div> <h1 style="margin: 0 0 8px 0; font-size: 28px; font-weight: 700; color: #164e63; letter-spacing: -0.5px; border: none; position: relative; z-index: 1;">Ling-3.0-flash-abliterated-APEX-GGUF</h1> <p style="margin: 8px 0 0 0; font-size: 14px; position: relative; z-index: 1;"><span style="color: #6b7280;">English</span> | <a href="https://huggingface.co/SC117/Ling-3.0-flash-abliterated-APEX-GGUF/blob/main/README_zh.md" style="color: #0891b2; text-decoration: none;">📖 中文文档</a></p> <p style="margin: 8px 0 0 0; font-size: 15px; color: #6b7280; position: relative; z-index: 1;">inclusionAI Ling-3.0-flash · 124B-A5.1B hybrid MoE · abliterix v6 trial 21 · APEX GGUF</p> </div> </div> <div style="font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif; display: flex; flex-direction: column; gap: 20px; margin-bottom: 30px;"> <div style="border: 1px solid #cbd5e1; border-radius: 12px; overflow: hidden; background: #ffffff; box-shadow: 0 2px 4px rgba(0,0,0,0.02);"> <div style="background: linear-gradient(135deg, #0891b2 0%, #0e7490 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>⚡</span> About this release</div> <div style="padding: 16px; font-size: 13px; color: #334155; line-height: 1.7;"> <p style="margin: 0 0 12px 0;"><a href="https://huggingface.co/inclusionAI/Ling-3.0-flash" target="_blank" style="color: #0e7490; text-decoration: none; font-weight: 700;">Ling-3.0-flash</a> is inclusionAI's next-generation native hybrid reasoning model: <b>124B total / 5.1B active</b> (~12.4% of their previous 1T-class flagship). It uses a native hybrid-linear stack from pretraining — <b>5:1 Kimi Delta Attention (KDA) + gated MLA</b>, 1/64 sparse MoE, 512 routed experts (top-8) + 1 shared expert, 2 dense layers, and a trained <b>MTP</b> head. Official context schedule is 8K → 32K → 256K. Thinking is on by default in the official card.</p> <p style="margin: 0 0 12px 0;">This community package is <b>not an official inclusionAI release</b>. It applies two extra steps on top of the official weights:</p> <ol style="margin: 0 0 12px 0; padding-left: 20px;"> <li style="margin-bottom: 6px;"><b>Refusal edit (mergeable recipe)</b> via <a href="https://github.com/wuwangzhang1216/abliterix" target="_blank" style="color: #0e7490; text-decoration: none; font-weight: 700;">abliterix</a> <b>v6 trial 21</b> LoRA, stream-merged back to BF16. Write path: MPOA <code>weight_normalization=full</code>, LoRA rank 3, <code>o_proj</code> + <code>down_proj</code> only. This is the static merge, <b>not</b> the later runtime-only gate (v36).</li> <li style="margin-bottom: 6px;"><b>GGUF</b>: BF16 convert + <a href="https://github.com/mudler/apex-quant" target="_blank" style="color: #0e7490; text-decoration: none; font-weight: 700;">APEX</a> mixed-precision tensor-type files (no imatrix on these tiers). Architecture in the GGUF is <code>bailingmoe3</code>.</li> </ol> <p style="margin: 0;"><b>License: MIT</b> (same family as the official model). Follow inclusionAI terms and local law.</p> </div> </div> <div style="border: 1px solid #fdba74; border-radius: 12px; overflow: hidden; background: #fff7ed; box-shadow: 0 2px 4px rgba(0,0,0,0.02);"> <div style="background: linear-gradient(135deg, #f97316 0%, #ea580c 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>⚠️</span> Abliterated notice</div> <div style="padding: 16px; font-size: 13px; color: #334155; line-height: 1.7;"> <p style="margin: 0 0 12px 0;">After merging abliterix <b>v6 trial 21</b>, this model shows a <b>much lower refusal rate</b> and can differ from official Ling-3.0-flash. Evaluate compliance and safety for your use case; control access and audit as needed.</p> <table style="width: 100%; border-collapse: collapse; font-size: 13px;"><tbody> <tr><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); font-weight: bold; color: #334155; background: white;">Refusals (search eval)</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #4b5563; background: white;"><b>17 / 98</b> (17.3%)</td></tr> <tr><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); font-weight: bold; color: #334155; background: white;">KL divergence</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #4b5563; background: white;">0.0933 nats/token</td></tr> <tr><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); font-weight: bold; color: #334155; background: white;">Selected trial</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #4b5563; background: white;">abliterix v6 trial 21 (mergeable). Not v36 (runtime gate).</td></tr> <tr><td style="padding: 6px 10px; font-weight: bold; color: #334155; background: white;">Compact smoke (thinking off)</td><td style="padding: 6px 10px; color: #4b5563; background: white;">Capability ~9/10; benign refuse 0/10; harmful refuse ~2–3/40. Ablation holds. Thinking-on eval can eat the token budget — use <code>--reasoning off</code> or a larger <code>max_tokens</code>.</td></tr> </tbody></table> <p style="margin: 12px 0 0 0; font-size: 12px; color: #9a3412;">Implementation sketch: LoRA merge <code>W += (B @ A) * (alpha / r)</code> with MPOA full-norm rank 3 on <code>o_proj</code> + <code>down_proj</code>. Adapter path: <code>Ling-3.0-flash-LoRA-Trial21-Refusals17-KL0.093</code>.</p> </div> </div> <div style="border: 1px solid #cbd5e1; border-radius: 12px; overflow: hidden; background: #ffffff; box-shadow: 0 2px 4px rgba(0,0,0,0.02);"> <div style="background: linear-gradient(135deg, #0891b2 0%, #0e7490 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>🧠</span> Model details</div> <div style="padding: 16px;"> <table style="width: 100%; border-collapse: collapse; font-size: 13px;"><tbody> <tr><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); font-weight: bold; color: #334155; background: white;">Architecture</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #4b5563; background: white;">Hybrid-linear MoE (<code>bailing_hybrid</code> / GGUF <code>bailingmoe3</code>)</td></tr> <tr><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); font-weight: bold; color: #334155; background: white;">Parameters</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #4b5563; background: white;">124B total, 5.1B active / token</td></tr> <tr><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); font-weight: bold; color: #334155; background: white;">Layers</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #4b5563; background: white;">35 KDA + 7 gated MLA (5:1) · 2 dense · GGUF 42 backbone + 1 MTP (<code>blk.42</code>)</td></tr> <tr><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); font-weight: bold; color: #334155; background: white;">Experts</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #4b5563; background: white;">512 routed + 1 shared, top-8</td></tr> <tr><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); font-weight: bold; color: #334155; background: white;">Attention</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #4b5563; background: white;">KDA (fine-grained diagonal gating) + gated MLA, 32 heads, hidden 2560</td></tr> <tr><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); font-weight: bold; color: #334155; background: white;">Context</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #4b5563; background: white;">Trained 8K → 32K → 256K (practical limit depends on VRAM / <code>-c</code>)</td></tr> <tr><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); font-weight: bold; color: #334155; background: white;">Vocab</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #4b5563; background: white;">157,184</td></tr> <tr><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); font-weight: bold; color: #334155; background: white;">Modality</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #4b5563; background: white;">Text → text (no mmproj)</td></tr> <tr><td style="padding: 6px 10px; font-weight: bold; color: #334155; background: white;">This repo</td><td style="padding: 6px 10px; color: #4b5563; background: white;">BF16 + APEX Compact / Quality / Balanced GGUF</td></tr> </tbody></table> <p style="margin: 12px 0 0 0; font-size: 12px; color: #64748b;">Coding / agent ability is largely retained on Compact smoke; refusal and alignment behavior are changed. Official SWE / Tau / MCP tables were not re-run for this derivative.</p> </div> </div> <div style="border: 1px solid #cbd5e1; border-radius: 12px; overflow: hidden; background: #ffffff; box-shadow: 0 2px 4px rgba(0,0,0,0.02);"> <div style="background: linear-gradient(135deg, #0891b2 0%, #0e7490 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>💡</span> What is APEX?</div> <div style="padding: 16px; font-size: 13px; color: #334155; line-height: 1.7;"> <p style="margin: 0 0 12px 0;">These files use <a href="https://github.com/mudler/apex-quant" target="_blank" style="color: #0e7490; text-decoration: none; font-weight: 700;">APEX</a>-style MoE-aware mixed precision: precision follows <b>tensor role + layer position</b> (higher on edges, more aggressive in the middle).</p> <p style="margin: 0;">This package is <b>not I-tier</b> (no imatrix). Common settings: 43 GGUF blocks (42 backbone + MTP <code>blk.42</code>); first 2 layers dense; routers <code>ffn_gate_inp=Q8_0</code>; <b>MTP + nextn.eh_proj forced Q8_0</b>; KDA/MLA tensor names as in llama.cpp <code>bailingmoe3</code>.</p> </div> </div> <div style="border: 1px solid #cbd5e1; border-radius: 12px; overflow: hidden; background: #ffffff; box-shadow: 0 2px 4px rgba(0,0,0,0.02);"> <div style="background: linear-gradient(135deg, #0891b2 0%, #0e7490 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>📦</span> Files</div> <div style="padding: 16px;"> <table style="width: 100%; border-collapse: collapse; font-size: 13px;"><thead><tr style="background: rgba(8,145,178,0.06);"> <th style="padding: 8px 10px; border-bottom: 2px solid #0891b2; text-align: left; color: #0e7490;">File</th> <th style="padding: 8px 10px; border-bottom: 2px solid #0891b2; text-align: left; color: #0e7490;">Size</th> <th style="padding: 8px 10px; border-bottom: 2px solid #0891b2; text-align: left; color: #0e7490;">Mid experts</th> <th style="padding: 8px 10px; border-bottom: 2px solid #0891b2; text-align: left; color: #0e7490;">Best for</th> </tr></thead><tbody> <tr><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); background: white;"><code>*-APEX-Quality.gguf</code></td><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); background: white;">75.90 GiB</td><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); background: white;">edge Q6_K / near Q5_K / mid <b>iq4_xs</b>; shared Q8_0; attn Q6_K</td><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); background: white;">Highest APEX quality (IQ mid-layers)</td></tr> <tr><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); background: #ecfeff;"><code>*-APEX-Balanced.gguf</code></td><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); background: #ecfeff;"><b>85.96 GiB</b></td><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); background: #ecfeff;">edge Q6_K / near &amp; mid <b>Q5_K</b>; shared Q8_0; attn Q6_K</td><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); background: #ecfeff;"><b>Recommended default</b> — steadier than Compact (no IQ mid)</td></tr> <tr><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); background: white;"><code>*-APEX-Compact.gguf</code></td><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); background: white;">56.25 GiB</td><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); background: white;">edge Q4_K / mid Q3_K; shared Q6_K; attn Q4_K</td><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); background: white;">Fits ~64–96 GB unified / VRAM with context room</td></tr> <tr><td style="padding: 8px 10px; background: white;"><code>*-BF16.gguf</code></td><td style="padding: 8px 10px; background: white;">237.57 GiB</td><td style="padding: 8px 10px; background: white;">native BF16 (norms / router / KDA state F32)</td><td style="padding: 8px 10px; background: white;">Requantize / archive. Too large for a 128 GB box.</td></tr> </tbody></table> <p style="margin: 12px 0 0 0; font-size: 13px; color: #334155; line-height: 1.7;"><b>How to choose:</b> start with <b>Balanced</b> if you have the RAM. Use <b>Compact</b> on 128 GB UMA (this is what we smoke-tested). <b>Quality</b> if you want IQ mid-experts and a smaller file than Balanced. Skip BF16 unless you are converting again.</p> </div> </div> <div style="border: 1px solid #cbd5e1; border-radius: 12px; overflow: hidden; background: #ffffff; box-shadow: 0 2px 4px rgba(0,0,0,0.02);"> <div style="background: linear-gradient(135deg, #0891b2 0%, #0e7490 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>🚀</span> Usage (llama.cpp)</div> <div style="padding: 16px; font-size: 13px; color: #334155; line-height: 1.7;"> <p style="margin: 0 0 8px 0;">These GGUFs are <code>general.architecture = bailingmoe3</code>. BailingMoE3 support (<a href="https://github.com/ggml-org/llama.cpp/pull/26608" target="_blank" style="color: #0e7490; text-decoration: none; font-weight: 700;">PR #26608</a>) was merged into llama.cpp master on 2026-08-17 — the first release containing it is <b>b10470</b>. Use llama.cpp <b>b10470 or newer</b> (any release or master build from that date onward) to load these files directly. If you see <code>unknown model architecture: 'bailingmoe3'</code>, your build is older than that — update. Do not mix with unofficial <code>bailing-hybrid</code> forks — the tensor names do not match.</p> <p style="margin: 0 0 8px 0; font-weight: bold; color: #164e63;">Example (Compact, thinking off)</p> <pre style="margin: 0 0 12px 0; font-family: monospace; background: #f8fafc; padding: 10px 14px; border-radius: 6px; border: 1px solid #e2e8f0; font-size: 12px; color: #1e293b; white-space: pre-wrap;">hf download SC117/Ling-3.0-flash-abliterated-APEX-GGUF \ --include "Ling-3.0-flash-abliterated-APEX-Compact.gguf" \ --local-dir ./models

./llama-server
-m ./models/Ling-3.0-flash-abliterated-APEX-Compact.gguf
--port 8080
-sm none
--flash-attn on
--fit on
--ctx-size 32768
--reasoning off
--jinja
--host 0.0.0.0</pre>

<ul style="margin: 0; padding-left: 20px;"> <li style="margin-bottom: 6px;">Official sampling: <code>temperature=0.6, top_p=0.95, top_k=20</code>. Official HF/SGLang/vLLM path enables thinking by default; for llama.cpp eval we used <code>--reasoning off</code> so thinking does not eat the token budget.</li> <li style="margin-bottom: 6px;">MTP is present as <code>blk.42</code>. This package was served with <code>--spec-type none</code>; speculative decode needs a build that understands Ling MTP.</li> <li style="margin-bottom: 0;">Text-only GGUF; no mmproj.</li> </ul> </div> </div> <div style="border: 1px solid #cbd5e1; border-radius: 12px; overflow: hidden; background: #ffffff; box-shadow: 0 2px 4px rgba(0,0,0,0.02);"> <div style="background: linear-gradient(135deg, #0891b2 0%, #0e7490 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>🎛️</span> Recommended sampling</div> <div style="padding: 16px;"> <table style="width: 100%; border-collapse: collapse; font-size: 13px;"><tbody> <tr><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); font-weight: bold; color: #334155; background: white;">Official default</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #4b5563; background: white; font-family: monospace;">temperature 0.6, top_p 0.95, top_k 20</td></tr> <tr><td style="padding: 6px 10px; font-weight: bold; color: #334155; background: white;">Thinking</td><td style="padding: 6px 10px; color: #4b5563; background: white;">Official: on by default. llama.cpp: <code>--reasoning on|off</code> as needed.</td></tr> </tbody></table> </div> </div> <div style="border: 1px solid #cbd5e1; border-radius: 12px; overflow: hidden; background: #ffffff; box-shadow: 0 2px 4px rgba(0,0,0,0.02);"> <div style="background: linear-gradient(135deg, #0891b2 0%, #0e7490 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>🔧</span> Build pipeline (summary)</div> <div style="padding: 16px; font-size: 13px; color: #334155; line-height: 1.7;"> <ol style="margin: 0; padding-left: 20px;"> <li style="margin-bottom: 6px;">Base: <code>inclusionAI/Ling-3.0-flash</code></li> <li style="margin-bottom: 6px;">abliterix v6 search → <b>trial 21</b> LoRA (<code>o_proj</code> + <code>down_proj</code>, MPOA rank 3)</li> <li style="margin-bottom: 6px;">BF16 merge → <code>Ling-3.0-flash-abliterated</code></li> <li style="margin-bottom: 6px;">llama.cpp PR #26608 convert → BF16 GGUF</li> <li style="margin-bottom: 0;">APEX tensor-type-file (no imatrix) → Compact / Quality / Balanced</li> </ol> </div> </div> </div>

Links

• Original model: https://huggingface.co/inclusionAI/Ling-3.0-flash
• inclusionAI: https://huggingface.co/inclusionAI
• ModelScope: https://modelscope.cn/organization/inclusionAI
• OpenRouter: https://openrouter.ai/inclusionai/ling-3.0-flash:free
• abliterix: https://github.com/wuwangzhang1216/abliterix
• APEX: https://github.com/mudler/apex-quant
• llama.cpp PR #26608 (bailingmoe3, merged 2026-08-17, first release b10470): https://github.com/ggml-org/llama.cpp/pull/26608

Disclaimer

Community derivative (behavior edit + quantization). Not an official inclusionAI release. Use at your own risk; follow local law and the upstream MIT license.

<!-- SC117_SUPPORT_START --> <div align="center"> <a href="https://ko-fi.com/sc117"> <img src="https://huggingface.co/SC117/Qwen3.8-Flash-Next-GSQ-RCO-abliterated-GGUF/resolve/main/assets/support-sc117.png" alt="Support SC117 on Ko-fi — support is optional and all releases remain freely available." width="100%" /> </a> </div> <!-- SC117_SUPPORT_END -->