Downloads · 30 days
496
73% of all-time downloads
GOBA-AI-Labs/PrunedHub-Qwen3-30B-A3B-EN-80pct-MxMoE
PrunedHub-Qwen3-30B-A3B-EN-80pct-MxMoE is a text generation model from GOBA-AI-Labs. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as apache-2.0.
13.45 GB | 102/128 experts | Mixed Q5K/Q4K/Q3K | -22% size, -4pp MMLU | GGUF | Apache 2.0
Downloads · 30 days
496
73% of all-time downloads
All-time downloads
678
Public
Repo size
14.4 GB
Likes
0
Public
Click a slice to open those files.
.gguf14.4 GB · 100%
From the Hugging Face model README
13.45 GB | 102/128 experts | Mixed Q5K/Q4K/Q3K | -22% size, -4pp MMLU | GGUF | Apache 2.0
A compressed variant of Qwen3-30B-A3B combining two techniques:
Optimized for English tasks. See JP variant for Japanese-optimized version.
| Benchmark | Original (17.28 GB) | This Model (13.45 GB) | Delta |
|---|---|---|---|
| MMLU (0-shot, 100Q, no-think) | 77% | 70% | -7 pp |
| MMLU STEM | -- | 57.5% | -- |
| MMLU Humanities | -- | 70.0% | -- |
| MMLU Social Sciences | -- | 86.7% | -- |
| GSM8K (0-shot, 50Q, no-think) | 92%* | 94% | +2 pp |
| Inference speed (M4 Pro, llama-server) | ~57 tok/s | ~57 tok/s | negligible |
| File size | 17.28 GB | 13.45 GB | -22.2% |
| Property | Value |
|---|---|
| Base model | Qwen/Qwen3-30B-A3B |
| Total parameters | 30B (3B active per token) |
| Architecture | Transformer + DeltaNet hybrid with Sparse MoE |
| MoE layers | 36 (+ 12 Attention layers) |
| Experts per layer | 102 (pruned from 128) |
| Routing | Top-8, softmax |
| Expert FFN dim | 768 (SwiGLU) |
| Shared expert FFN dim | 6144 |
| Context length | 32K tokens |
| Quantization | MxMoE -- Q5K (12 layers) / Q4K (24 layers) / Q3K (12 layers) |
| Pruning | EN-calibrated, 80% expert retention |
| File size | 13.45 GB |
| License | Apache 2.0 |
# Run with llama-server (Metal GPU, ~57 tok/s)
llama-server \
-m Qwen3-30B-A3B-pruned80-EN-MxMoE.gguf \
--port 8090 \
-ngl 99 \
-c 4096
Also compatible with moe-stream for MoE-optimized inference:
git clone https://github.com/GOBA-AI-Labs/moe-stream
cd moe-stream && cargo build --release --features accelerate
./target/release/moe-stream-server \
--model Qwen3-30B-A3B-pruned80-EN-MxMoE.gguf \
--port 11434
Once llama-server or moe-stream-server is running:
from openai import OpenAI
client = OpenAI(base_url="http://localhost:11434/v1", api_key="none")
response = client.chat.completions.create(
model="default",
messages=[{"role": "user", "content": "Explain quantum entanglement in simple terms."}],
max_tokens=512,
temperature=0.7,
)
print(response.choices[0].message.content)
Each expert is scored using calibration-based importance, computed from actual inference on representative English text:
importance(l, e) = activation_magnitude(l, e) × routing_frequency(l, e)
The bottom 20% of experts per layer are removed (128 → 102 experts/layer). This model uses English-calibrated importance scores, meaning experts are selected to maximize English task performance.
After pruning, each layer's expert FFN tensors are quantized at different precision levels based on layer importance:
| Tier | Layers | Quant | Rationale |
|---|---|---|---|
| Tier 1 (top 25%) | 12 layers | Q5_K | Critical layers -- high importance + routing frequency |
| Tier 2 (middle 50%) | 24 layers | Q4_K | Standard precision |
| Tier 3 (bottom 25%) | 12 layers | Q3_K | Redundant layers -- low importance |
Constraints:
mean_expert_importance × mean_routing_frequencyOriginal Q4_K_M: 17.28 GB (128 experts × 48 layers)
After 80% pruning: ~15.0 GB (102 experts × 48 layers)
After MxMoE: 13.45 GB (Q5K/Q4K/Q3K mixed)
Total reduction: -22.2%
| Keep % | Experts/Layer | Size | MMLU | Notes |
|---|---|---|---|---|
| 100% (original) | 128 | 17.28 GB | 77% | Baseline |
| 90% | 115 | 15.6 GB | 75% | Minor loss |
| 80% (this model) | 102 | 13.45 GB* | 70% | Best size-quality tradeoff |
| 70% | 90 | 12.3 GB | 51% | Quality cliff |
| 60% | 77 | -- | Collapse | Unusable |
* With MxMoE; 80% pruning alone = ~15.0 GB at Q4_K_M uniform.
The pruning scripts and plans are available at GOBA-AI-Labs/moe-prune:
# 1. Download Qwen3-30B-A3B Q4_K_M GGUF
# 2. Generate importance scores
python scripts/qwen3_30b_importance.py
# 3. Apply language-aware pruning (80% keep, EN-calibrated)
python scripts/reprune_gguf.py \
Qwen3-30B-A3B-Q4_K_M.gguf \
Qwen3-30B-A3B-pruned80-EN.gguf \
results/calibration/qwen3_30b_a3b/plan_80pct_en_natural.json
# 4. Apply MxMoE mixed quantization
llama-quantize --allow-requantize \
--tensor-type-file results/mxmoe/tensor_types_pruned.txt \
Qwen3-30B-A3B-pruned80-EN.gguf \
Qwen3-30B-A3B-pruned80-EN-MxMoE.gguf q4_k_m
@misc{goba2026moe,
title = {Language-Aware Expert Pruning with Mixed Quantization for MoE Language Models},
author = {GOBA-AI-Labs},
year = {2026},
url = {https://github.com/GOBA-AI-Labs/moe-prune},
note = {Calibration-based pruning + MxMoE achieves 22\% size reduction on Qwen3-30B-A3B}
}
llama-quantize toolThis model inherits the Apache 2.0 License from the base model (Qwen/Qwen3-30B-A3B).