Downloads · 30 days
41
2% of all-time downloads
Litwein/Qwen3.6-35B-A3B-Claude-4.7-Opus-DWQ4-MTP-MLX
Qwen3.6-35B-A3B-Claude-4.7-Opus-DWQ4-MTP-MLX is a text generation model from Litwein. Use it when you need the model to write or continue text. It is set up for mlx. The card lists the license as apache-2.0.
[](https://huggingface.co/Litwein/Qwen3.6-35B-A3B-Claude-4.7-Opus-oQ4e-DWQ-MC-MTP-MLX)
Downloads · 30 days
41
2% of all-time downloads
All-time downloads
2K
Public
Parameters
35.5B
20 GB on disk
Likes
2
Public
Click a slice to open those files.
.safetensors20 GB · 100%
How the weights are stored.
U3235.5B · 100%
From the Hugging Face model README
⚠️ Superseded — use v2 instead
👉
Qwen3.6-35B-A3B-Claude-4.7-Opus-oQ4e-DWQ-MC-MTP-MLXv2 is a two-stage imatrix → DWQ build (reasoning, then math/code) of the same base. It is materially better on code — HumanEval 90.2 % (this v1 line: 84.8 %) · LiveCodeBench 39.9 % (v1: 32 %) · MBPP 75.4 % — at the same size class, same MTP packaging.
This v1 repo is kept for reproducibility only. New users should download v2.
A 4-bit MLX quantization (DWQ) of
lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled — text-only, with MTP speculative decoding.
This is a text build of the reasoning-distilled Qwen3.6-35B-A3B, packaged for fast local inference on Apple Silicon with an embedded Multi-Token-Prediction (MTP) head for speculative decoding.
⚠️ These are quantized weights. All capability comes from the base model
lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled— please star/cite it first. This repo's contribution is the quantization + MTP packaging and its fidelity/speed validation.
Qwen/Qwen3.6-35B-A3B (Apache-2.0 base, 35B MoE · 256 experts · ~3B active · 262k ctx)
└─ lordx64/…-Claude-4.7-Opus-Reasoning-Distilled (SFT distillation of Claude Opus 4.7 reasoning)
└─ THIS REPO: DWQ 4-bit MLX + MTP
<think>…</think> reasoning. Attention-only LoRA (r=16) via Unsloth + TRL.This is a DWQ quant — Apple MLX's learned quantization
(mlx-lm LEARNED_QUANTS).
A uniform 4-bit student is gradient-distilled to match an 8-bit teacher of the same
model (KL divergence on the top-1024 logits, temperature 2.0, lr 1e-6), so the flat 4-bit
weights recover most of the fidelity that plain round-to-nearest 4-bit loses.
| Scheme | affine uniform 4-bit, group size 64 (router gate / shared_expert_gate at 8-bit, the mlx-lm default) |
| Effective size | ~4.3 bits/weight — uniform, so fast dense matmuls |
| Distillation teacher | 8-bit MLX quant of the same base |
| Calibration | 2,856 pre-split ≤512-token windows (reasoning tails covered, not just heads) |
| Result | validation KL-to-teacher 0.0334 → 0.0325 (improved) over the undistilled 4-bit start |
| Tooling | mlx_lm.quant.dwq (MLX 0.31.x) |
Scope: this is the DWQ stage of MLX's
Dynamic → DWQrecipe. The student is a uniform 4-bit convert (round-to-nearest), not a Dynamic-quant mixed-precision student — so fidelity is recovered by distillation, not by per-layer bit allocation. The sibling oQ4 repos are the sensitivity-mixed (Dynamic-style) quant on their own; a fullDynamic → DWQbuild (mixed precision then distillation) is a natural next step.
Because DWQ produces a uniform 4-bit model, it is both smaller and slightly faster than sensitivity-mixed schemes at equal task quality (see Evaluation).
Two kinds of numbers below: quant-fidelity metrics measured on this quant, and capability benchmarks inherited from the bf16 base (a 4-bit quant preserves — does not create — capability; the fidelity metrics show how little was lost).
| Metric | This model | Notes |
|---|---|---|
| Gradeable math (10 short-answer problems) | 10 / 10 | arithmetic, GCD, algebra, calculus, combinatorics — all complete within budget |
Code (4 tasks, exec-checked) | 4 / 4 | RLE, median, balanced-brackets, int→Roman |
| Perplexity | 2.75 (held-out reasoning text, lower = better) | |
| Decode speed (MTP on, M4 Pro 48 GB) | ~54–56 tok/s | multi-token-prediction speculative decoding (measured) |
| MTP draft acceptance | ~40–64 % | content-dependent |
Note on long-form math. This is a reasoning model that routinely emits 8,000+ tokens of
<think>on olympiad problems. Fixed-budget pass/fail on such problems is token-budget-limited, not quantization-limited — every 4-bit and 8-bit level behaves the same. Give it a largemax_tokens(32k) for hard problems.
Reported by the base model lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled via lm-evaluation-harness; listed here as a
capability reference for the family, not re-measured on this quant.
| Benchmark | Base (bf16) |
|---|---|
| GSM8K (8-shot CoT) | 84.3 % |
| MMLU-Pro (5-shot) | 74.9 % |
| MMLU-Pro · Math | 83.6 % |
| MMLU-Pro · Physics | 81.0 % |
| MMLU-Pro · Computer Science | 79.0 % |
| Repo | Quant | Vision | Size | Best for |
|---|---|---|---|---|
…-DWQ4-MTP-Vision-MLX | DWQ uniform 4-bit | ✓ | ~21 GB | smallest+fastest, multimodal |
…-DWQ4-MTP-MLX | DWQ uniform 4-bit | – | ~19 GB | smallest+fastest, text only |
…-oQ4-MTP-Vision-MLX | oq mixed 4–8-bit | ✓ | ~22 GB | mixed-precision, multimodal |
…-oQ4-MTP-MLX | oq mixed 4–8-bit | – | ~21 GB | mixed-precision, text only |
TL;DR: the DWQ builds are smaller, a touch faster, and match the oQ builds on task quality — distillation lets uniform 4-bit keep up with mixed precision. oQ builds spend extra bits on sensitive layers (a more conservative, classic dynamic quant).
These are MLX weights (Apple Silicon). The tested serving path is oMLX, which supports this model's native MTP speculative decoding out of the box.
Text-only (no vision tower) — leanest footprint for pure text + MTP.
1. Place the folder in your oMLX models directory:
# after downloading this repo:
mv Qwen3.6-35B-A3B-Claude-4.7-Opus-DWQ4-MTP-MLX ~/.omlx/models/
2. Enable MTP for this model (one-time), then call the OpenAI-compatible API:
curl -X PUT http://127.0.0.1:8003/admin/api/models/qwen3.6-35b-a3b-claude-4.7-opus-dwq4-mtp-mlx/settings \
-H "Authorization: Bearer sk-local" -H "Content-Type: application/json" \
-d '{"mtp_enabled": true}'
curl -X POST http://127.0.0.1:8003/v1/chat/completions \
-H "Authorization: Bearer sk-local" -H "Content-Type: application/json" \
-d '{"model": "qwen3.6-35b-a3b-claude-4.7-opus-dwq4-mtp-mlx",
"messages": [{"role": "user", "content": "Prove there are infinitely many primes."}],
"max_tokens": 8000, "temperature": 0.6}'
The uniform 4-bit backbone also loads directly in stock mlx-lm as a qwen3_5_moe text model (verified).
MTP speculative decoding needs an MTP-aware runtime — oMLX (tested), or mlx-lm with PR #990.
temperature 0.6, top_p 0.95, top_k 20. For hard reasoning set max_tokens ≥ 32000 — the
model thinks in explicit <think>…</think> blocks and can be long.
<think> blocks — budget tokens accordingly.lordx64/reasoning-distill-opus-4-7-max-sft
— ~7,800 Claude Opus 4.7 reasoning conversations.mlx, mlx-lm (DWQ · mlx_lm.quant.dwq), mlx-vlm.oq quantizer, MTP serving runtime, and OpenAI-compatible API.Apache-2.0, inherited from the base model and Qwen3.6-35B-A3B.
@misc{qwen36_a3b_2026, title={Qwen3.6-35B-A3B}, author={Qwen Team}, year={2026},
howpublished={\url{https://huggingface.co/Qwen/Qwen3.6-35B-A3B}} }
@misc{lordx64_qwen36_distill_2026,
title={Qwen3.6-35B-A3B distilled from Claude Opus 4.7 reasoning}, author={lordx64}, year={2026},
howpublished={\url{https://huggingface.co/lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled}} }