Qwen3.6-35B-A3B-6bit-XL-mlx

Flight Simulator — Coded by This Model

A real-world coding benchmark: each model was prompted to write a complete flight simulator from scratch. The resulting code was rendered and recorded.

About XL Quantization

BaseQuant_XL is a fully data-agnostic, static quantization. No calibration dataset, no sensitivity analysis, no importance matrix. Precision is allocated purely by architectural role — routing-critical layers get higher precision, bulk expert parameters get lower precision. The result is a transparent, faithful capture of the source model.

Data-dependent calibration quantizations (iMatrix, AWQ, GPTQ, oQ, oQ4e, etc.) use a calibration set to guide bit allocation. This can produce a skewed representation of the model: domains well-represented in the calibration data (English, popular topics, public or leaked benchmarks) are preserved better, while underrepresented domains (non-English languages, niche use cases, your own data) are preserved worse. XL avoids this trade-off entirely — it generalizes honestly because it is never fit to any particular data distribution.

Thinking-Mode MATHQA: Gemma vs Qwen3.6 Token Efficiency

Both models were tested on the corrected custom MATHQA test set (n=30, thinking mode, 8192-token budget). Both verified at 30/30 (100%) after re-test of failures. The decisive difference is how efficiently they arrive at correct answers.

Gemma uses 29% fewer reasoning chars per correct answer than Qwen3.6 (verified results after re-test of failures). Gemma's shorter reasoning chains mean:

• Less context window consumed — more space for multi-turn conversation or complex prompts
• Less energy per answer — fewer GPU cycles wasted on self-verification
• More decisive reasoning — Gemma commits to answers sooner; Qwen3.6 tends to verify and re-derive

The per-question scatter shows Gemma's reasoning clustered at 1K–4K chars while Qwen3.6 spreads from 2K–12K. On the hardest questions (Q2232, Q2932), both models spike to 11K–19K chars — the efficiency gap closes when the problem genuinely requires extended reasoning.

⚠️ Raw accuracy differences are dominated by sampling variance (temp=1.0) and answer-extraction artifacts. Both models are capable of 30/30. The real differentiator is reasoning efficiency — correct answers per thousand reasoning chars.

Local SOTA for 48GB Macs — Intelligence Benchmark Comparison

This model is part of a benchmark comparison of the best local MLX-quantized LLMs that fit in 48GB unified memory on Apple Silicon. All benchmarks run in instruct mode (no thinking) with n=50 samples per benchmark.

Collection: Local SOTA for 48GB Macs

⚠️ n=50 sampling means wide confidence intervals (±~13% at 95% CI). Differences under ~6 points may not be statistically significant. Models using data-aware quantization (oQ/oQe) may be calibrated on benchmark-like data — their scores carry a benchmaxxing caveat. The BaseQuant_XL variants (data-agnostic) provide the most honest generalization estimates.

leonsarmiento/Qwen3.6-35B-A3B-6bit-XL-mlx

This model was converted to MLX format from Qwen/Qwen3.6-35B-A3B using BaseQuant_XL 6/8-bit mixed quantization optimized for Apple Silicon. The vision encoder is preserved and quantized at 6-bit, making this a full multimodal model.

BaseQuant_XL keeps the most routing-critical layers in full bf16 precision — the MoE router gate, shared expert gate, shared expert, and lm_head — while applying aggressive quantization to the bulk parameters. This preserves routing accuracy and output quality where it matters most.

Qwen3.6-35B-A3B is a 35B-parameter multimodal MoE (Mixture of Experts) model with 256 experts (8 active per token + 1 shared expert), hybrid full + linear (Gated DeltaNet) attention, and a vision encoder. Despite 35B total parameters, only ~3B are activated per token for efficient inference.

Intelligence Benchmarks (Instruct mode, n=50)

Analysis: The Honest King

This model uses BaseQuant_XL — a fully data-agnostic quantization. No calibration data, no sensitivity analysis, no iMatrix. Layers are quantized to fixed bit depths based purely on their architectural role. The results above reflect genuine generalization quality, not calibration fit.

The competing oQ/oQe variants use data-aware quantization (oQ: sensitivity analysis, oQe: iMatrix-like process). Their scores may be benchmaxxed — if the calibration dataset overlaps with these benchmarks, their higher scores reflect calibration quality as much as model quality.

Key observations:

1. This model beats oQ6-mtp across all 5 benchmarks (74.8% vs 71.2% average) despite both being 6-bit. This is remarkable — the data-agnostic XL quantization outperforms the data-aware oQ at the same bit depth. It suggests oQ's sensitivity-based calibration at 6-bit may be counterproductive for this architecture, potentially overfitting certain weight regions at the expense of others.

2. MMLU_PRO is tied for the best at 64.0% — matching the oQ4e-mtp-XL Frankenstein and beating both data-aware-only variants by +4. Since MMLU_PRO tests multi-step reasoning across 120 subjects, this is the benchmark least susceptible to calibration overfitting and most representative of genuine reasoning ability.

3. HUMANEVAL (78%) and MBPP (78%) are honest, solid results. They can't match the data-aware oQ4e variants (90%+) — but those scores may reflect calibration on code-like data. If your use case differs from the calibration set, the XL's 78% is a more trustworthy expectation.

4. The Frankenstein oQ4e-mtp-XL leads on average (78.8%) by combining XL's data-agnostic attention with oQ4e's data-aware experts. Its MMLU score of 72% (highest of all variants) likely reflects the XL attention layers' routing quality — but the coding scores still carry calibration risk from the oQ4e experts.

⚠️ n=50 sampling means wide confidence intervals (±~13% at 95% CI). Differences under ~6 points may not be statistically significant. The ranking should be interpreted directionally, not definitively. The key takeaway is the philosophical difference: data-aware quants optimize for the calibration set, data-agnostic quants optimize for generalization. Choose based on whether your workload matches the calibration data.

Custom MATHQA (thinking mode, n=30, corrected test set)

A failure-enriched MATHQA test set (n=30) run with thinking enabled and a relaxed 8192-token budget to isolate reasoning quality from throughput artifacts. The test set was manually audited and 9 of 19 "hard" questions corrected for dataset errors (dropped digits, wrong expected answers, impossible constraints) before this run — see methodology in leonsarmiento/gemma-4-26B-A4B-it-6bit-XL-mlx.

Results: 6-bit family + 4-bit cross-reference

Raw vs verified accuracy: The "verified" column reflects a re-test of all questions that failed on either model — all models solved all 30 questions on re-run. Original failures were transient: sampling-induced degenerate loops (20K+ char spirals with zero extractable output) and answer-extraction artifacts (markdown bold formatting missed by regex). See re-test details.

Key findings

1. XL reduces reasoning waste. The XL variant uses fewer reasoning chars per correct answer than the non-XL at the same bit depth — this model achieves the lowest reasoning waste of all variants tested (5,111 chars/correct). The bf16 routing precision produces more decisive reasoning chains that reach confidence thresholds sooner, without the self-verification spirals common in lower-precision routing. This finding replicates across both bit depths and on Gemma 4 26B A4B.

2. 6-bit is more decisive than 4-bit. Both 6-bit models scored 96.7% raw vs 90.0% for both 4-bit models. The 6-bit experts produce sufficient intermediate precision for the model to commit to answers — fewer degenerate loops, fewer extraction failures, less reasoning waste per correct answer.

3. XL trades throughput for efficiency. This model is the slowest in raw tok/s (46.8) but compensates with fewer tokens generated overall (68K vs 72K for oQ6-mtp). The net wall-time difference is modest (+11%). The XL's advantage is context-window efficiency — less reasoning means more space for actual answers, multi-turn conversation, or long context.

4. MTP is not recommended for thinking mode. While the 6-bit family did not show MTP-related accuracy degradation (both scored 96.7%), the 4-bit XL with MTP enabled dropped to 83.3% raw accuracy (+20% wall time, 3 degenerate loops). See the MTP evaluation on the oQ4e-XL model card. MTP's throughput boost is wasted when the model controls its own output length — faster tokens lead to longer reasoning, not faster answers.

Re-test details

Two questions failed in the initial run across both 6-bit models (1 each):

• Q636 (oQ6-mtp): Degenerate loop — 159s with zero extractable output. Re-test: solved correctly in 34s with 5,743 reasoning chars.
• Q2232 (6bit-XL): Extraction artifact — 13s with only 261 reasoning chars, answer missed by regex. Re-test: solved correctly in 25s with 231 reasoning chars.

⚠️ Thinking mode uses temperature=1.0, which introduces sampling variance. Individual questions can occasionally spiral into degenerate loops on any model. The n=30 raw scores carry ±10% noise; the re-test confirms both models are capable of perfect scores on this test set.

Use with mlx

pip install -U mlx-vlm

python -m mlx_vlm.generate --model leonsarmiento/Qwen3.6-35B-A3B-6bit-XL-mlx --max-tokens 256 --temperature 0.6 --top-p 0.95 --prompt "Hello"

BaseQuant_XL Quantization Strategy

Quantization Details

• Quantization type: BaseQuant_XL mixed (multimodal, vision preserved)
• Bits per weight: 6.808
• Total size: ~28 GB (6 shards)
• Group size: 64
• Method: Custom quant_predicate via mlx_vlm

Recommended Inference Parameters - Add to Jinja template on LM studio or Chat Template Kwargs on oMLX

Thinking Preserve ({%- set preserve_thinking = true %}):

• General tasks: temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0
• Coding tasks: temperature=0.6, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, repetition_penalty=1.0

Instruct Mode ({%- set enable_thinking = false -%}):

• General tasks: temperature=0.7, top_p=0.8, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0
• Reasoning tasks: temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0

**Note:**This model does not have MTP layers — it is recommended for thinking-mode workloads where reasoning stability matters more than raw token throughput.