Downloads · 30 days
150
1% of all-time downloads
sandeshrajx/Qwen3.5-24B-A3B-REAP-0.32-GGUF
Qwen3.5-24B-A3B-REAP-0.32-GGUF is a text generation model from sandeshrajx. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
<p align="center" <em𓌳 <strongREAP</strong𓌳 the Experts: Why Pruning Prevails for One-Shot MoE Compression</em<br </p
Downloads · 30 days
150
1% of all-time downloads
All-time downloads
13.8K
Public
Repo size
73.1 GB
Likes
13
Public
Click a slice to open those files.
.gguf72.9 GB · 100%
From the Hugging Face model README
Introducing Qwen3.5-24B-A3B-REAP-0.32, a memory-efficient compressed variant of Qwen3.5-35B-A3B that maintains the core reasoning and coding capabilities of the architecture while being 32% lighter.
This model was created using REAP (Router-weighted Expert Activation Pruning), a novel expert pruning method that selectively removes redundant experts while preserving the router's independent control over remaining experts. Key features include:
transformers (from source) and vLLM.Qwen3.5-24B-A3B-REAP-0.32 has the following specifications:
This repository contains the following artifacts:
Qwen3.5-24B-A3B-REAP-0.32-IQ4_K_M.gguf: High-precision 4-bit quant using the Unsloth-style recipe (imatrix + Q8_0 overrides for critical tensors).Qwen3.5-24B-A3B-REAP-0.32-IQ4_K_S.gguf: Smaller 4-bit quant variant.Qwen3.5-24B-A3B-REAP-0.32-Q4_K_M.gguf: Naive Q4_K_M quant.imatrix.dat: The importance matrix used for quantization.calibration_data_v5_rc.txt: The calibration corpus used to generate the imatrix.Since Qwen 3.5 MoE is a new architecture, ensure you are using the latest transformers from source:
pip install git+https://github.com/huggingface/transformers.git
You can deploy the model directly using vLLM:
vllm serve sandeshrajx/Qwen3.5-24B-A3B-REAP-0.32 \
--enable-expert-parallel
Optimized GGUF versions are available in this repository. We recommend using the IQ4_K_M variant for the best balance of size and performance.
REAP selects experts to prune based on a saliency criterion that considers router gate values and expert activation norms. This ensures that only experts contributing minimally to the model's internal representations are removed.
The project utilized Modal for high-memory compute (A100-80GB) and a custom fork of the REAP library.
This model is derived from Qwen3.5-35B-A3B and distributed under the Apache 2.0 License.
@article{lasby-reap,
title={REAP the Experts: Why Pruning Prevails for One-Shot MoE compression},
author={Lasby, Mike and Lazarevich, Ivan and Sinnadurai, Nish and Lie, Sean and Ioannou, Yani and Thangarasa, Vithursan},
journal={arXiv preprint arXiv:2510.13999},
year={2025}
}