Qwen3.6-29B-REAP-Opus-Reasoning-Distill-MLX-3bit

Qwen3.6-29B-REAP-Opus-Reasoning-Distill-MLX-3bit

MLX 3-bit conversion of barozp/Qwen3.6-29B-REAP-Opus-Reasoning-Distill-MTP -- the MTP-grafted variant of the REAP-pruned (205/256 experts) Qwen/Qwen3.6-35B-A3B MoE fine-tuned on Claude Opus reasoning traces -- for Apple Silicon. Sibling formats:

• bf16 safetensors source: barozp/Qwen3.6-29B-REAP-Opus-Reasoning-Distill-MTP
• plain merge (no MTP head): barozp/Qwen3.6-29B-REAP-Opus-Reasoning-Distill
• GGUF quantizations: barozp/Qwen3.6-29B-REAP-Opus-Reasoning-Distill-GGUF
• other MLX quant: barozp/Qwen3.6-29B-REAP-Opus-Reasoning-Distill-MTP-MLX-4bit

Converted straight from the bf16 safetensors weights (not from any GGUF quant).

Conversion details

• Bits: 3 (group_size=64, affine) — experimental quality tier
• Command: python -m mlx_lm.convert --hf-path barozp/Qwen3.6-29B-REAP-Opus-Reasoning-Distill-MTP --mlx-path <out> --quantize --q-bits 3 --q-group-size 64
• Tooling: mlx 0.32.1 · mlx-lm 0.31.3 · hub 1.27.0; source commit 9fa24b516dd3
• Converter: mlx-lm
• Environment: Google Colab G4 (RTX PRO 6000 Blackwell 96 GB), MLX cuda12 backend
• Full conversion log ships in this repo as conversion_log.txt

MTP head note

The source checkpoint carries an MTP head (mtp_num_hidden_layers: 1). Standard MLX loaders do not use it: this conversion dönüştürücü tarafından düşürüldü (beklenen davranış; MTPLX build'i için bkz. mtplx forge). These builds therefore behave identically to the plain-merge checkpoint at inference time. For native-MTP speculative decoding on Apple Silicon, build an MTPLX artifact from the source repo with mtplx forge (github.com/youssofal/MTPLX).

Usage

pip install -U mlx-lm

# text-only chat
python -m mlx_lm.generate \
  --model barozp/Qwen3.6-29B-REAP-Opus-Reasoning-Distill-MTP-MLX-3bit \
  --prompt "Explain the Monty Hall problem briefly." \
  --max-tokens 512

# OpenAI-compatible local server
python -m mlx_lm.server \
  --model barozp/Qwen3.6-29B-REAP-Opus-Reasoning-Distill-MTP-MLX-3bit

About the source model

A REAP-pruned (205/256 experts) Qwen3.6-35B-A3B MoE (~28.24B total, ~3B active per token), LoRA fine-tuned on Claude Opus reasoning traces, then merged. Key result from the source card: ARC-Challenge above the unpruned base (0.616 vs 0.532) with 96.9% MMLU retention at 20% expert pruning. Method paper: arXiv:2510.13999.

Source chain

Qwen/Qwen3.6-35B-A3B (base MoE) -> RangerX/Qwen3.6-35B-REAP-Pruned-ratio-0.2 (REAP pruning) -> barozp/Qwen3.6-29B-REAP-Opus-Reasoning-Distill (LoRA finetune, merged) -> barozp/Qwen3.6-29B-REAP-Opus-Reasoning-Distill-MTP (MTP head graft) -> this repo (MLX 3-bit conversion)