Ling-3.0-flash-REAP176-46B-A5B-GGUF

This is an experimental REAP.

Ling-3.0-flash REAP176 (46B total / 5.1B active) - GGUF

[176 of 512 routed experts kept per layer - 65.6% of experts pruned; 176 = 22 expert groups of 8, the group-size-divisible step nearest the 174 target] from inclusionAI/Ling-3.0-flash.

🚨 This is essentially a lobotomized model and it does not work as expected. 🚨

It remains public in case anyone is interested in using it for experimentation or testing. This was the most heavily pruned REAP of the bunch that was performed against Ling 3.0 Flash.

The following REAPs are less degraded and likely worth testing (if your hardware allows):

• https://huggingface.co/bloomer010/Ling-3.0-flash-REAP384-97B-A5B-GGUF
• https://huggingface.co/bloomer010/Ling-3.0-flash-REAP320-81B-A5B-GGUF
• https://huggingface.co/bloomer010/Ling-3.0-flash-REAP288-73B-A5B-GGUF

Method: one-shot REAP (Router-weighted Expert Activation Pruning) - experts scored by router-gate-value × output-L2-norm over calibration data, lowest-scoring deleted. No fine-tuning, no recovery training.

Calibration: 1M tokens, 50/25/25 ultrachat / wikitext / code

🎉 bailingmoe3 is supported in stock llama.cpp since PR #26608 (merged 2026-08-17, commit 3733366720). Any build from that commit onward loads these files directly.

🔔 2026-08-21: added reasoning_effort support (low = thinking off, high = on, default same). If you want reasoning_effort," re-download or override with chat_template.jinja.

Serving with experts in CPU RAM (attention on GPU, experts streamed from RAM):

llama-server -m Ling-3.0-flash-REAP176-45B-A5B-MXFP4.gguf \
  -ngl 99 -ot "ffn_.*_exps\.weight=CPU" --no-mmap -c 65536 --flash-attn on --jinja

Quants in this repo (all cut from the full-precision BF16 export): MXFP4, Q4_K_M, Q3_K_M, Q2_K

• MXFP4 (experts MXFP4 / rest Q8_0) is the pick for CPU-offload serving. Tiers upload as they are cut; check the file list for current availability.