Downloads · 30 days
9
18% of all-time downloads
qwen3-30b-a3b-td-moe-ep-fix
AverageMetaheuristicsEnjoyer/qwen3-30b-a3b-td-moe-ep-fix
qwen3-30b-a3b-td-moe-ep-fix is a text generation model from AverageMetaheuristicsEnjoyer. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
TD-MoE Tucker decomposition on Qwen3-30B-A3B, compression matched to MoBE/Sparse-MoBE b32 (37.5% removed per gate/up matrix, 62.5% kept). Expert-preserving.
All-time downloads
50
Public
Parameters
23.3B
93.2 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors46.6 GB · 100%
At a glance
- Task
- Text Generation
- License
- apache-2.0
- Model type
- qwen3_moe
- Access
- Public
- Created
- Jun 3, 2026
- Updated
- Jun 3, 2026
- SHA
- eaa17c86
Try a prompt
Base models
Model card
From the Hugging Face model README
td_moe_qwen3_30b_a3b_match_mobe_ep
TD-MoE Tucker decomposition on Qwen3-30B-A3B, compression matched to MoBE/Sparse-MoBE b32 (37.5% removed per gate/up matrix, 62.5% kept). Expert-preserving.
- Method:
td_moe - Base model: Qwen/Qwen3-30B-A3B
- Compressed:
gate_proj, up_projover layers 0-48 (down_projkept dense) - Settings: TD-MoE Tucker, compression_ratio=0.375 (removed), expert_mode=preserve
See config.toml and training.json for the full run configuration and per-layer reconstruction stats.
Evaluation
| Task | Metric | Value |
|---|---|---|
| arc_challenge | sample_len | 1172.0000 |
| arc_challenge | acc,none | 0.3311 |
| arc_challenge | acc_stderr,none | 0.0138 |
| arc_challenge | acc_norm,none | 0.3626 |
| arc_challenge | acc_norm_stderr,none | 0.0140 |
| arc_easy | sample_len | 2376.0000 |
| arc_easy | acc,none | 0.5724 |
| arc_easy | acc_stderr,none | 0.0102 |
| arc_easy | acc_norm,none | 0.5101 |
| arc_easy | acc_norm_stderr,none | 0.0103 |
| winogrande | sample_len | 1267.0000 |
| winogrande | acc,none | 0.6259 |
| winogrande | acc_stderr,none | 0.0136 |
| piqa | sample_len | 1838.0000 |
| piqa | acc,none | 0.6779 |
| piqa | acc_stderr,none | 0.0109 |
| piqa | acc_norm,none | 0.7051 |
| piqa | acc_norm_stderr,none | 0.0106 |
| hellaswag | sample_len | 10042.0000 |
| hellaswag | acc,none | 0.3995 |
| hellaswag | acc_stderr,none | 0.0049 |
| hellaswag | acc_norm,none | 0.5269 |
| hellaswag | acc_norm_stderr,none | 0.0050 |
| openbookqa | sample_len | 500.0000 |
| openbookqa | acc,none | 0.2680 |
| openbookqa | acc_stderr,none | 0.0198 |
| openbookqa | acc_norm,none | 0.3340 |
| openbookqa | acc_norm_stderr,none | 0.0211 |
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained(
"AverageMetaheuristicsEnjoyer/qwen3-30b-a3b-td-moe-ep-fix",
trust_remote_code=True,
torch_dtype="bfloat16",
device_map="auto",
)
tok = AutoTokenizer.from_pretrained("AverageMetaheuristicsEnjoyer/qwen3-30b-a3b-td-moe-ep-fix", trust_remote_code=True)
The factored experts are reconstructed at runtime via the bundled Qwen3TDMoEForCausalLM.
- Task
- Text Generation
- Type
- qwen3_moe
- License
- apache-2.0
- Created
- Jun 3, 2026
- Updated
- Jun 3, 2026