Downloads · 30 days
7
8% of all-time downloads
Qwen3-Coder-30B-A3B-REAM-AWQ
mattbucci/Qwen3-Coder-30B-A3B-REAM-AWQ
Qwen3-Coder-30B-A3B-REAM-AWQ is a machine learning model from mattbucci. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
REAM-pruned and AWQ-quantized variant of Qwen/Qwen3-Coder-30B-A3B-Instruct, built end-to-end from the upstream BF16 base. 128 experts merged down to 96 (~30B → ~23B total params; 3B active retained), then 4-bit AWQ-qu…
All-time downloads
92
Public
Parameters
23.3B
13 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors13 GB · 100%
Parameter types
How the weights are stored.
I3222.6B · 97%
Base models
Model card
From the Hugging Face model README
Qwen3-Coder-30B-A3B-REAM AWQ 4-bit
REAM-pruned and AWQ-quantized variant of Qwen/Qwen3-Coder-30B-A3B-Instruct, built end-to-end from the upstream BF16 base. 128 experts merged down to 96 (~30B → ~23B total params; 3B active retained), then 4-bit AWQ-quantized for AMD RDNA4 (gfx1201) inference with SGLang.
Model Details
| Base model | Qwen/Qwen3-Coder-30B-A3B-Instruct (upstream BF16, 128 experts) |
| Architecture | Qwen3 MoE (96 experts post-REAM, top-8) |
| Parameters | ~23B total / 3B active |
| Pruning | REAM (Router Expert Aware Merging) — Samsung SAIL merge.py with saliency=reap, grouping=ream, merging=logits+weights, mix_ratio=0.0,0.3,0.7. REAM merges similar experts into representative survivors (it does not drop experts the way Cerebras REAP does). |
| Layers | 48 |
| Context | 256K (max_position_embeddings=262144) |
| Quantization | Native AWQ 4-bit, group_size=128, GEMM kernel format |
| Calibration | llmcompressor GPTQ, 256 samples × 1024 tokens, code+thinking mix; ignore=['lm_head'] (router mlp.gate is preserved BF16 — targets=Linear skips the router) |
The full REAM merge args are captured in config.json["merge_args"] for reproducibility.
Usage with SGLang
The recommended path on RDNA4 (gfx1201) is --quantization moe_wna16 --dtype bfloat16. Per-Linear AWQ + fp16 routes 128 experts × 48 layers × 3 projections through individual GEMV kernels per forward, which is much slower (and on some configurations triggers a HSA exception). The MoE-fused kernel is the fast and stable path.
# vLLM
from vllm import LLM
llm = LLM(model="mattbucci/Qwen3-Coder-30B-A3B-REAM-AWQ",
quantization="moe_wna16", dtype="bfloat16")
# SGLang (CLI)
python -m sglang.launch_server \
--model-path mattbucci/Qwen3-Coder-30B-A3B-REAM-AWQ \
--quantization moe_wna16 --dtype bfloat16
For other inference engines, this is a standard AWQ 4-bit checkpoint (group_size=128, asymmetric, fused MoE) and should load via transformers + autoawq without modification.
Notes
- REAM ≠ REAP. REAM (Samsung SAIL) merges experts; REAP (Cerebras
cerebras/Qwen3-Coder-REAP-25B-A3B) drops them. Different algorithms with different artifacts; not interchangeable. - The
shared_expert_gate(a Linear with output dim 1) is preserved in BF16 — AWQ's group-quantization packing requires output dim divisible by 8.
Hardware
REAM-merged + calibrated + smoke-tested on 2× AMD Radeon AI PRO R9700 (gfx1201, RDNA4, 64 GB total VRAM) with ROCm 7.2 and SGLang v0.5.10/v0.5.11.
Source code for the build pipeline (RDNA4 patches, calibration scripts, conversion utilities): https://github.com/mattbucci/2x-R9700-RDNA4-GFX1201-sglang-inference
- Type
- qwen3_moe
- License
- apache-2.0
- Created
- May 9, 2026
- Updated
- May 9, 2026