DeepSeek-V4-Flash-192t2-UD-IQ1_S

DeepSeek-V4-Flash-192t2 (UD-IQ1_S, per-layer expert pruning)

Hugging Face: siendsi/DeepSeek-V4-Flash-192t2-UD-IQ1_S

A 25% expert-pruned build of DeepSeek-V4-Flash-0731 (UD-IQ1_S), reduced from 82.4 GB → 60.0 GB (−22.4%) by keeping the 192 most-activated experts per layer (out of 256) and disabling the remaining 192.

This is a GGUF model for llama.cpp. It is a per-layer MoE expert pruning of the original — not a low-rank distillation and not a weight re-quantization. All non-expert weights and the kept experts are bit-identical to the original UD-IQ1_S checkpoint.

What was done

• Method: per-layer expert pruning. For each of the 43 MoE layers, the 192 experts with the highest activation counts (from a calibration "heat" map) are kept; the 192 least-used experts are disabled.
• Calibration heat: tuned to a target profile — Russian + English + coding + planning + agentic tasks, plus a math corpus (math is critical for agentic/reasoning tasks). The original calibration corpus (which included Japanese/Chinese) was not used, because it pruned experts that matter for the target profile.
• Quantization: unchanged — UD-IQ1_S (Unsloth dynamic quant).
• Architecture: deepseek4 (llama.cpp).

Why per-layer? DeepSeek-V4 expert IDs are per-layer — expert N in layer 0 is a different expert than expert N in layer 3 (0/256 shared). A global top-K expert map is therefore meaningless; pruning must be done independently per layer.

Size

Quality (A/B vs. original)

Evaluated against the original UD-IQ1_S on a target-profile battery:

• Basic battery (10 tasks): RU/EN knowledge, RU/EN math, code, planning, agent — 10/10 parity with the original.
• Deep battery (24 tasks): code / devops / multi-hop / OS / NLP / agent — 23/24 parity.
• Extended battery (16 tasks): RU/EN knowledge, math, logic, code, planning, agent, translation — 16/16 parity.

Known limitation

One real degradation was found: on a "write a minimal Dockerfile" task the pruned model can loop in its reasoning (finish=length, empty content, even with a 2000-token budget), whereas the original answers correctly. Simple code/NLP/multi-hop/OS/agent tasks are unaffected.

Practical takeaway: for short, deterministic answers (knowledge, math, code snippets, NLP) this model is at parity with the original. For tasks that require long, self-correcting reasoning, keep-192 may fail to converge.

Speed (llama.cpp, -t 8, clean measurement)

Prefill is noticeably faster on the pruned model; generation is roughly at parity (the number of active top-k experts is the same, so the difference in weight loading is minimal).

Usage (llama.cpp)

llama-server \
  -m DeepSeek-V4-Flash-192t2-UD-IQ1_S-00001-of-00003.gguf \
  -c 32768 -t 8 -fa on --load-mode mmap \
  --port 8085

Or with the llama CLI:

llama-cli -m DeepSeek-V4-Flash-192t2-UD-IQ1_S-00001-of-00003.gguf \
  -c 32768 -t 8 -fa on --load-mode mmap

The model is multi-shard. Point llama.cpp at shard 00001; it will pick up the other shards automatically.

Files

Provenance

• Base: DeepSeek-V4-Flash-0731 (UD-IQ1_S, Unsloth dynamic quant)
• License: MIT (inherited from the base model)
• Pruning: per-layer top-192/256 experts, target-profile + math heat
• Built with: llama.cpp rebuild_gguf.py (per-layer prune config)
• Reproduction: see the glm-pruning project — DSV4_PRUNING_FINAL.md, tools/build_dsv4_prune_config_perlayer.py, tools/heat_target.py

Notes

• This is a pruned model, not a fine-tune. Behavior on the target profile is at parity with the original; out-of-profile (e.g. Japanese/Chinese) is not the goal and may be degraded.
• Very long contexts (>32k) were not stress-tested.