deepseek_r1_distilled_qwen_7B_sparse_50

Model: DeepSeek-R1-Distill-Qwen-7B Sparse 50%

Base model: deepseek-ai/DeepSeek-R1-Distill-Qwen-7B

Method: SparseGPT unstructured pruning at 50% sparsity

Calibration data: GSM8K math problems (128 samples)

Sparsity achieved: 42.95%

Quality: Multi-step reasoning preserved. Model correctly solves word problems with step-by-step explanation.

Hardware used: Kaggle T4 GPU

Purpose: Proof of concept for PE-MoE architecture — demonstrating that reasoning quality survives 50% weight pruning on a distilled reasoning model.

Limitations:

Sparsity is unstructured — requires sparse-aware inference for memory benefits

Calibration data was general, not domain-specific

Not quantized — AWQ step not yet applied

42.95% actual sparsity vs 50% target due to layer-wise variation

Quality tested on simple math only, not comprehensive benchmarks