Qwen3.6-35B-PreREAP-BNB8-Pruned-ratio-0.3

Qwen3.6-35B-A3B Pre-REAP BNB8 Pruned Ratio 0.3

This checkpoint is a routed-expert-pruned version of Qwen/Qwen3.6-35B-A3B. Expert saliency was collected with REAP while loading the scoring model with bitsandbytes 8-bit quantization for standard linear layers. The final saved checkpoint is the original BF16 model pruned according to those quantization-aware scores.

Pruning setup

• Base model: Qwen/Qwen3.6-35B-A3B
• Method: REAP routed-expert pruning
• Pre-REAP scoring model quantization: bitsandbytes 8-bit
• Pruning ratio: 0.30
• Calibration samples: 1024
• Sequence length: 2048
• Seed: 42
• Router renormalization: enabled
• Shared experts: preserved

Notes

This model uses the packed Qwen3.5/Qwen3.6 MoE integration in the REAP codebase. During bnb8 scoring, standard nn.Linear modules are quantized by bitsandbytes, while packed routed-expert tensors remain BF16 parameters. The checkpoint itself is saved after pruning in BF16 format and can be loaded with Transformers using trust_remote_code=True.

Evaluation is still in progress for this specific bnb8-pruned checkpoint. Prior comparison runs use plain lm-eval prompts for GSM8K, without chat templating.