Downloads · 30 days
120
8% of all-time downloads
leonsarmiento/Qwen3.6-35B-A3B-oQ4e-mtp-XL-mlx
Qwen3.6-35B-A3B-oQ4e-mtp-XL-mlx is a image-text-to-text model from leonsarmiento. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for mlx. The card lists the license as apache-2.0.
A real-world coding benchmark: each model was prompted to write a complete flight simulator from scratch. The resulting code was rendered and recorded.
Downloads · 30 days
120
8% of all-time downloads
All-time downloads
1.5K
Public
Parameters
36B
22.7 GB on disk
Likes
1
Public
Click a slice to open those files.
.safetensors22.7 GB · 100%
How the weights are stored.
U3234.9B · 97%
From the Hugging Face model README
A real-world coding benchmark: each model was prompted to write a complete flight simulator from scratch. The resulting code was rendered and recorded.

BaseQuant_XL is a fully data-agnostic, static quantization. No calibration dataset, no sensitivity analysis, no importance matrix. Precision is allocated purely by architectural role — routing-critical layers get higher precision, bulk expert parameters get lower precision. The result is a transparent, faithful capture of the source model.
Data-dependent calibration quantizations (iMatrix, AWQ, GPTQ, oQ, oQ4e, etc.) use a calibration set to guide bit allocation. This can produce a skewed representation of the model: domains well-represented in the calibration data (English, popular topics, public or leaked benchmarks) are preserved better, while underrepresented domains (non-English languages, niche use cases, your own data) are preserved worse. XL avoids this trade-off entirely — it generalizes honestly because it is never fit to any particular data distribution.
This model is part of a benchmark comparison of the best local MLX-quantized LLMs that fit in 48GB unified memory on Apple Silicon. All benchmarks run in instruct mode (no thinking) with n=50 samples per benchmark.

| Benchmark | Samples | Agents-A1 6bit-XL | Gemma-4 26B 6bit-XL | Huihui-Qwen3.6 6bit-XL | Ornith-35B 6bit-XL | Qwen3.6-27B oQ4e | Qwen3.6-35B 6bit-XL | Qwen3.6-35B oQ4e | Qwen3.6-35B oQ4e-XL | Qwen3.6-35B oQ6 |
|---|---|---|---|---|---|---|---|---|---|---|
| MMLU | 50/14042 | 66% | 76% | 74% | 64% | 74% | 64% | 66% | 72% | 64% |
| MMLU_PRO | 50/12032 | 58% | 82% | 66% | 66% | 56% | 64% | 60% | 64% | 60% |
| ARC_CHALLENGE | 50/1172 | 90% | 90% | 92% | 92% | 88% | 90% | 92% | 92% | 90% |
| HUMANEVAL | 50/164 | 90% | 98% | 84% | 78% | 92% | 78% | 92% | 90% | 66% |
| MBPP | 50/500 | 70% | 82% | 78% | 78% | 86% | 78% | 80% | 76% | 76% |
| Average | 74.8% | 85.6% | 78.8% | 75.6% | 79.2% | 74.8% | 78.0% | 78.8% | 71.2% |
Collection: Local SOTA for 48GB Macs
⚠️ n=50 sampling means wide confidence intervals (±~13% at 95% CI). Differences under ~6 points may not be statistically significant. Models using data-aware quantization (oQ/oQe) may be calibrated on benchmark-like data — their scores carry a benchmaxxing caveat. The BaseQuant_XL variants (data-agnostic) provide the most honest generalization estimates.
⚡ oMLX REQUIRED: These quants are designed to be used with oMLX and they will not work on LM studio. The bundled MTP (Lightning MTP) speculative-decoding heads only execute on oMLX — LM Studio loads the base model but ignores the MTP heads, so the acceleration is lost. For MTP speed, serve this model with oMLX.
A hybrid MLX quantization that surgically combines layers from two quantization variants of Qwen3.6-35B-A3B:
leonsarmiento/Qwen3.6-35B-A3B-5bit-XL-mlx — BaseQuant_XL 8-bit / bf16 (data-agnostic)switch_mlp), MTP layer, vision tower from Jundot/Qwen3.6-35B-A3B-oQ4e-mtp — oQ4e 4-bit (data-aware)The result is a quantization-level layer swap — not a weight-space merge (DARE-TIES, Task Arithmetic). The model architecture is identical; only the quantization precision and method per layer differs.
The two donor quantizations use fundamentally different approaches:
| Aspect | XL (BaseQuant_XL) | oQ4e-mtp (oQ Quantization) |
|---|---|---|
| Method | Data-agnostic uniform group quantization | Data-aware — calibration dataset guides bit allocation |
| Attention | 8-bit, group_size=64 | Mixed 5/6/8-bit (data-aware) |
| Experts | 5-bit, group_size=64 | 4-bit, group_size=64 |
| Shared Expert Gate | bf16 (unquantized) | 8-bit |
| Shared Expert | bf16 (unquantized) | 8-bit |
| lm_head | bf16 (unquantized) | 4-bit |
| Vision Tower | 5-bit | bf16 (unquantized) |
| MTP Layer | Not present | Present (8-bit) |
The hypothesis: XL's high-precision 8-bit attention and bf16 shared expert/lm_head provide the best token routing and output projection, while oQ4e's data-aware 4-bit experts preserve the weight regions that matter most for the oQ calibration data. The MTP (Multi-Token Prediction) layer from oQ4e-mtp is preserved for speculative decoding support.
Important consideration: oQ/oQe quantization is data-aware — its quality depends on how well the oQ calibration dataset represents your use case. If your workload aligns with the calibration data, data-aware experts can outperform data-agnostic quantization at the same or lower bit depth. If it doesn't, the data-agnostic XL quantization may be more reliable. This hybrid gives you data-aware experts with data-agnostic attention — a middle ground.
| Benchmark | Samples | oQ4e-mtp-XL (this) | oQ4e-mtp | oQ6-mtp | 6bit-XL |
|---|---|---|---|---|---|
| MMLU | 50/14042 | 72.0% | 66.0% | 64.0% | 64.0% |
| MMLU_PRO | 50/12032 | 64.0% | 60.0% | 60.0% | 64.0% |
| ARC_CHALLENGE | 50/1172 | 92.0% | 92.0% | 90.0% | 90.0% |
| HUMANEVAL | 50/164 | 90.0% | 92.0% | 66.0% | 78.0% |
| MBPP | 50/500 | 76.0% | 80.0% | 76.0% | 78.0% |
| Average | 78.8% | 78.0% | 71.2% | 74.8% |
This comparison spans three quantization philosophies:
Key observations:
This model scores highest on average (78.8%), edging out oQ4e-mtp (78.0%) by +0.8pp. The XL attention layers boost MMLU by +6 (72 vs 66) and MMLU_PRO by +4 over oQ4e-mtp — knowledge recall benefits from higher-precision routing. However, the data-aware experts from oQ4e likely contribute to the coding benchmark strength.
The benchmaxxing caveat is critical. oQ/oQe quantizations use calibration data that may include these exact benchmarks (or similar distributions). If so, their scores are partly a measure of calibration fit, not general intelligence. The XL quantization carries no such risk — its results are honest.
If we discount data-aware results as potentially benchmaxxed, the 6bit-XL is the honest king. Its average of 74.8% reflects pure architectural quality without calibration assistance. Notably, it matches or beats oQ6-mtp (71.2%) across all five benchmarks — the 6-bit data-aware variant is actually weaker than data-agnostic XL, suggesting oQ calibration at 6-bit may be counterproductive for this model.
HUMANEVAL tells the most interesting story. oQ4e-mtp leads at 92.0%, this model follows at 90.0%, but oQ6-mtp drops to 66.0% — a 26-point collapse. This suggests data-aware 4-bit calibration is highly effective for code generation, but data-aware 6-bit calibration may actively harm it. The data-agnostic 6bit-XL lands in between at 78.0% — a safe, honest result.
⚠️ n=50 sampling means wide confidence intervals (±~13% at 95% CI). Differences under ~6 points may not be statistically significant. The HUMANEVAL gap between oQ6-mtp (66%) and the 4-bit variants (90%+) is too large to dismiss as noise, but most other differences are within sampling error.
A failure-enriched MATHQA test set (n=30) run with thinking enabled and a relaxed 8192-token budget to isolate reasoning quality from throughput artifacts. The test set was manually audited and 9 of 19 "hard" questions corrected for dataset errors (dropped digits, wrong expected answers, impossible constraints) before this run — see methodology in leonsarmiento/gemma-4-26B-A4B-it-6bit-XL-mlx.
| Model | MTP | Accuracy (raw) | Accuracy (verified) | Wall Time | Throughput | Reasoning Chars / Correct |
|---|---|---|---|---|---|---|
| oQ4e-mtp | ON | 27/30 (90.0%) | 30/30 (100%) | 1,285s | 73.8 tok/s | 6,881 |
| oQ4e-mtp-XL (this) | OFF | 27/30 (90.0%) | 30/30 (100%) | 1,396s | 48.5 tok/s | 5,847 |
| oQ4e-mtp-XL (this) | ON | 25/30 (83.3%) | — | 1,679s | 58.5 tok/s | 6,200 |
Raw vs verified accuracy: The "raw" column reflects the initial benchmark run. The "verified" column reflects a re-test of all 5 questions that failed on either model — both models solved all 5 correctly on re-run. The original failures were:
**E**) or non-standard answer formatting that the extraction regex missed. On re-test with an improved regex, all 3 were recovered as correct.Key findings:
⚠️ Thinking mode uses temperature=1.0, which introduces sampling variance. Individual questions can occasionally spiral into degenerate loops on any model. The n=30 raw scores carry ±10% noise; the re-test confirms both models are capable of perfect scores on this test set.
⚠️ Note on MTP and TurboQuant when using oMLX: Tests on oMLX 0.5.5 show no MTP (Lightning MTP) advantage when TurboQuant KV Cache is turned On. For MTP decoding speed advantages, TurboQuant should be disabled.
This model preserves the MTP (Multi-Token Prediction) layer from the oQ4e-mtp donor for speculative decoding support in compatible runtimes (LM Studio, oMLX). MTP can significantly boost raw token generation throughput — on this model, from ~48 tok/s (MTP-OFF) to ~58 tok/s (MTP-ON), a +21% throughput increase.
However, our evaluation reveals an important caveat for thinking/reasoning mode:
| Metric | MTP-OFF | MTP-ON | Delta |
|---|---|---|---|
| Accuracy | 27/30 (90.0%) | 25/30 (83.3%) | −2 questions |
| Total wall time | 1,396s | 1,679s | +20% slower |
| Throughput | 48.5 tok/s | 58.5 tok/s | +21% faster |
| Degenerate loops | 0 | 3 | 144–147s spirals with zero output |
The problem: In thinking mode, the model is an autonomous reasoner — it decides how long to think. MTP's faster token generation doesn't produce the same answer sooner; instead, the model thinks more with the extra speed. This creates three possible outcomes:
Recommendation:
| Layer | Source | Bits | Group Size | Method |
|---|---|---|---|---|
mlp.gate (router) | XL | bf16 | — | — |
shared_expert_gate | XL | bf16 | — | — |
lm_head | XL | bf16 | — | — |
shared_expert.up_proj / down_proj | XL | bf16 | — | — |
embed_tokens | XL | 8 | 64 | Data-agnostic |
self_attn (full attention) | XL | 8 | 64 | Data-agnostic |
linear_attn (DeltaNet) | XL | 8 | 64 | Data-agnostic |
shared_expert.gate_proj | XL | 8 | 64 | Data-agnostic |
switch_mlp (routed experts) | oQ4e-mtp | 4 | 64 | Data-aware |
vision_tower | oQ4e-mtp | bf16 | — | — |
| MTP layer | oQ4e-mtp | 8 | 64–128 | Data-aware |
Qwen3.6-35B-A3B is a 35B-parameter multimodal MoE (Mixture of Experts) model with 256 experts (8 active per token + 1 shared expert), hybrid full + linear (Gated DeltaNet) attention, and a vision encoder. Despite 35B total parameters, only ~3B are activated per token for efficient inference.
pip install -U mlx-vlm
python -m mlx_vlm.generate --model leonsarmiento/Qwen3.6-35B-A3B-oQ4e-mtp-XL-mlx --max-tokens 256 --temperature 0.6 --top-p 0.95 --prompt "Hello"
Thinking Preserve ({%- set preserve_thinking = true %}):
temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0temperature=0.6, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, repetition_penalty=1.0Instruct Mode ({%- set enable_thinking = false -%}):
temperature=0.7, top_p=0.8, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0**Note:**If your runtime supports MTP, keep it disabled for thinking-mode workloads — see the MTP evaluation section above.