Qwen3.6-35B-A3B-MXFP8-MTP

JANGQ-AI/Qwen3.6-35B-A3B-MXFP8-MTP

Qwen3.6-35B-A3B-MXFP8-MTP is a image-text-to-text model from JANGQ-AI. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for mlx. The card lists the license as apache-2.0.

<p align="center"<img src="osaurus-x-banner.png" width="100%" alt="OsaurusAI"/</p

Downloads · 30 days

108

6% of all-time downloads

All-time downloads

1.8K

Public

Parameters

36B

37.5 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors37.5 GB · 100%

Parameter types

How the weights are stored.

U3235.5B · 99%

Try a prompt

Base models

Qwen/Qwen3.6-35B-A3B

Model card

From the Hugging Face model README

Qwen3.6-35B-A3B-MXFP8-MTP

Qwen3.6-35B-A3B quantized to native MXFP8 for Apple Silicon, with the vision tower and the native Multi-Token-Prediction head preserved and enabled.


Source	Qwen/Qwen3.6-35B-A3B
License	Apache-2.0, inherited from upstream
Format	MXFP8 (`mx.quantize`, affine, `group_size=32`)
Architecture	`qwen3_5_moe` — 40 layers, 256 routed experts, top-8, ~3B active
Modality	image + video + text
Context	262,144
Bundle size	34.95 GB
MTP	native head preserved, enabled (`num_nextn_predict_layers=1`)

Quantization

8-bit affine linears via MLX-native mx.quantize (mode="mxfp8", group_size=32). Norms, router gates, expert biases and the full vision tower are kept in fp16 passthrough (643 passthrough tensors). MTP linears are quantized to MXFP8; MTP norm/control tensors stay fp16.

Multi-Token Prediction

This bundle keeps Qwen3.6's native MTP module and runs it as a self-speculative draft head: the MTP head proposes tokens that the main model verifies in a single pass, so decoded output stays bit-identical to plain autoregressive decoding — only faster.

Recorded on an M5 Max (vMLX runtime, 96-token deterministic prompt, output verified equal to baseline at every depth):

Draft depth	tok/s	Speedup
Baseline (MTP off)	59.4	1.00×
D1	86.9	1.46×
D2	98.5	1.66×
D3 (default)	101.6	1.71×

With vMLX prefix/KV cache layers enabled the speedup holds — a recorded cache-on A/B measured 59.8 → 88.8 tok/s (1.48×).

Absolute tok/s depends on free memory and system load. The speedup ratio — baseline vs. MTP measured back-to-back under identical conditions — is the stable figure.

Vision, MTP and caching together

These bundles run image/video input, native MTP speculative decode and prefix/KV caching in the same session — a combination not every MTP-enabled Qwen build exposes. A recorded VL probe (2026-05-16) confirms a color identification image prompt returns the correct answer through the combined MTP + VL runtime.

Loading

Loads via stock MLX tooling on Apple Silicon — the mxfp8 weights are native mx.quantize affine, no JANG runtime required for the core model.

from mlx_vlm import load, generate
model, processor = load("OsaurusAI/Qwen3.6-35B-A3B-MXFP8-MTP")

The MTP draft path is exercised by an MTP-aware runtime (vMLX); other runtimes load and decode the main model normally and ignore the MTP head.

Variants

Variant	Arch	Format	Size	Best MTP speedup
Qwen3.6-27B-MXFP4-MTP	dense	mxfp4	14.4 GB	1.85× (D2)
Qwen3.6-27B-MXFP8-MTP	dense	mxfp8	27.1 GB	1.83× (D3)
Qwen3.6-35B-A3B-MXFP4-MTP	MoE	mxfp4	21.5 GB	1.56× (D3)
Qwen3.6-35B-A3B-MXFP8-MTP (this)	MoE	mxfp8	35.0 GB	1.71× (D3)

Credits

Quantization toolchain: JANG by Jinho Jang <[email protected]>
Base model: Qwen3.6-35B-A3B by Qwen

Task: Image-Text-to-Text
Library: mlx
Type: qwen3_5_moe
License: apache-2.0
Created: May 17, 2026
Updated: May 17, 2026