Downloads · 30 days
139
11% of all-time downloads
Litwein/Qwen3.6-35B-A3B-Claude-4.7-Opus-oQ4e-DWQ-MC-MTP-Vision-MLX
Qwen3.6-35B-A3B-Claude-4.7-Opus-oQ4e-DWQ-MC-MTP-Vision-MLX is a image-text-to-text model from Litwein. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for mlx. The card lists the license as apache-2.0.
⭐ v2 — the best build of this model we ship Two-stage imatrix → DWQ (reasoning, then math/code). Materially better on code than the v1 repos: HumanEval 90.2 % (v1-stage: 84.8 %) · LiveCodeBench 39.9 % (32 %) · MBPP 75…
Downloads · 30 days
139
11% of all-time downloads
All-time downloads
1.3K
Public
Parameters
36B
21.6 GB on disk
Likes
3
Public
Click a slice to open those files.
.safetensors21.6 GB · 100%
How the weights are stored.
U3235.5B · 99%
From the Hugging Face model README
⭐ v2 — the best build of this model we ship
Two-stage imatrix → DWQ (reasoning, then math/code). Materially better on code than the v1 repos: HumanEval 90.2 % (v1-stage: 84.8 %) · LiveCodeBench 39.9 % (32 %) · MBPP 75.4 %. If you used any of the four
…-DWQ4-…/…-oQ4-…v1 repos, use this instead.
A 4-bit MLX quantization of
lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled —
vision + text, with an embedded MTP head for speculative decoding.
⚠️ These are quantized weights. All capability comes from the base model — please star/cite it first. This repo's contribution is the quantization recipe + MTP/vision packaging and its fidelity/benchmark validation.
Qwen/Qwen3.6-35B-A3B (Apache-2.0 base · 35B MoE · 256 experts · ~3B active · 262k ctx)
└─ lordx64/…-Claude-4.7-Opus-Reasoning-Distilled (SFT distill of Claude Opus 4.7 reasoning)
└─ THIS REPO: oQ4e (imatrix) → DWQ¹ reasoning → DWQ² math/code + MTP + vision
<think>…</think>
reasoning.oQ4e (imatrix) → two-stage DWQNot a plain round-to-nearest 4-bit. Three steps:
oQ4e — imatrix mixed precision. oMLX's enhanced quantizer builds an importance
matrix from calibration activations (sized for MoE expert coverage) and allocates bits by
sensitivity. Base 4-bit affine, group size 64, with sensitive tensors promoted:
196 → 8-bit, 114 → 5-bit, 1 → 6-bit.oQ8e (imatrix 8-bit) teacher on reasoning calibration (KL over the teacher's top-1024
logits, temperature 2.0, Adam, cosine LR, validation early-stopping, finalized from the
best checkpoint).| Scheme | affine 4-bit base (g64) · imatrix-promoted 5/6/8-bit · two DWQ passes |
| Effective size | ~4.5 bits/weight → 21.6 GB on disk (incl. bf16 vision tower + MTP head) |
| Distillation teacher | oQ8e — imatrix 8-bit quant of the same base |
| Calibration | leak-free, document-level splits, head/middle/tail ≤512-token windows |
— stage 1 (reasoning-v2) | 2,048 train / 128 valid — Opus-4.7 reasoning traces |
— stage 2 (benchmark-gap-math-code-swe-v4) | 2,048 train / 128 valid — verified math, competitive programming, concise-function code, resolved SWE trajectories + a small reasoning replay partition to limit forgetting |
| Tooling | oMLX oq (enhanced) + mlx_lm.quant.dwq |
Why two stages: stage 1 recovers general fidelity to the 8-bit teacher; stage 2 re-tunes the 4-bit grid toward code/math activations where the v1 builds were weakest. The reasoning replay partition in stage 2 keeps the reasoning behaviour from drifting.
| Benchmark | v1 (reasoning-DWQ stage) | v2 — this repo |
|---|---|---|
| HumanEval (pass@1) | 84.8 % | 90.2 % |
| MBPP | 74.8 % | 75.4 % |
| LiveCodeBench | 32 % | 39.9 % |
Run by the uploader on this 4-bit build via oMLX. Reported to show that the recipe preserves and re-tunes base capability, not as official base-model results — confirm harness/shot settings against your own eval before citing.
Reported by the base model via lm-evaluation-harness; a capability reference for the family,
not re-measured on this quant.
| Benchmark | Base (bf16) |
|---|---|
| GSM8K (8-shot CoT) | 84.3 % |
| MMLU-Pro (5-shot) | 74.9 % |
| MMLU-Pro · Math | 83.6 % |
| MMLU-Pro · Physics | 81.0 % |
| MMLU-Pro · Computer Science | 79.0 % |
Note on long-form math. This is a reasoning model that routinely emits 8,000+ tokens of
<think>on hard problems. Fixed-budget pass/fail there is token-budget-limited, not quantization-limited. Give it a largemax_tokens(32k) for hard problems.
| Repo | Stage | Vision | Size | |
|---|---|---|---|---|
…-oQ4e-DWQ-MC-MTP-Vision-MLX (this) | v2 · imatrix→DWQ×2 | ✓ | ~21.6 GB | best · multimodal |
…-oQ4e-DWQ-MC-MTP-MLX | v2 · imatrix→DWQ×2 | – | ~20.7 GB | best · text-only |
…-DWQ4-MTP-Vision-MLX, …-DWQ4-MTP-MLX, …-oQ4-MTP-Vision-MLX, …-oQ4-MTP-MLX | v1 | ✓/– | ~19–22 GB | superseded — kept for reproducibility |
Both v2 repos share the same distilled backbone + MTP head (byte-identical); they differ only by the vision tower — identical task quality.
MLX weights (Apple Silicon). Tested serving path is oMLX ≥ 0.5.0,
which supports this model's native MTP speculative decoding and vision out of the box.
This repo includes the vision tower (vision_tower.*, bf16) — it accepts images as well as text.
# 1. place the folder in your oMLX models directory
mv Qwen3.6-35B-A3B-Claude-4.7-Opus-oQ4e-DWQ-MC-MTP-Vision-MLX ~/.omlx/models/
# 2. enable MTP once, then call the OpenAI-compatible API
curl -X PUT http://127.0.0.1:8003/admin/api/models/Qwen3.6-35B-A3B-Claude-4.7-Opus-oQ4e-DWQ-MC-MTP-Vision-MLX/settings \
-H "Authorization: Bearer sk-local" -H "Content-Type: application/json" \
-d '{"mtp_enabled": true}'
curl -X POST http://127.0.0.1:8003/v1/chat/completions \
-H "Authorization: Bearer sk-local" -H "Content-Type: application/json" \
-d '{"model": "Qwen3.6-35B-A3B-Claude-4.7-Opus-oQ4e-DWQ-MC-MTP-Vision-MLX",
"messages": [{"role": "user", "content": "Write a Python LRU cache with tests."}],
"max_tokens": 8000, "temperature": 0.6}'
The backbone also loads directly in stock mlx-lm as a qwen3_5_moe text model. MTP
speculative decoding needs an MTP-aware runtime — oMLX (tested). Note the oMLX model id is
case-sensitive (it matches the folder name).
temperature 0.6, top_p 0.95, top_k 20, min_p 0. For hard reasoning set max_tokens ≥ 32000 —
the model thinks in explicit <think>…</think> blocks. Avoid greedy (temperature 0): reasoning
models degenerate there.
mlx, mlx-lm (mlx_lm.quant.dwq), mlx-vlm.oq/oQe imatrix quantizer, MTP + vision serving runtime.Apache-2.0, inherited from the base model and Qwen3.6-35B-A3B.
@misc{qwen36_a3b_2026, title={Qwen3.6-35B-A3B}, author={Qwen Team}, year={2026},
howpublished={\url{https://huggingface.co/Qwen/Qwen3.6-35B-A3B}} }
@misc{lordx64_qwen36_distill_2026,
title={Qwen3.6-35B-A3B distilled from Claude Opus 4.7 reasoning}, author={lordx64}, year={2026},
howpublished={\url{https://huggingface.co/lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled}} }