Qwen3.6-35B-A3B-MLX-Mixed-4-8

Qwen3.6-35B-A3B MLX Mixed 4/8-bit

An MLX mixed-precision quantization of unsloth/Qwen3.6-35B-A3B, itself built on the official Qwen/Qwen3.6-35B-A3B. Quantized for inference on Apple Silicon using the mlx-lm framework.

Model overview

Qwen3.6-35B-A3B is a Mixture-of-Experts (MoE) language model from the Qwen3.5 family. The "35B" refers to total parameters; "A3B" means only ~3 billion parameters are active per forward pass — the model routes each token through 8 of its 256 experts, making inference dramatically cheaper than a dense 35B model.

Architecture note: This model uses Qwen3VLProcessor and carries vision/audio special token IDs, meaning the architecture is multimodal-capable. This quantization is intended for text-generation use with mlx-lm.

Quantization details

This is a mixed 4/8-bit quantization using the MLX affine scheme (group_size: 64). Sensitive layers (embeddings, attention projections, linear attention layers) are kept at 8-bit; the bulk of the MoE expert weights are compressed to 4-bit.

Disk size: ~20 GB across 4 safetensors shards.

Hardware requirements

This model requires Apple Silicon with the MLX framework. Due to the MoE architecture, active memory use is much lower than total parameter count suggests.

The 256K context window is the theoretical maximum; practical context at 24 GB will be shorter.

Usage

Install dependencies:

pip install mlx-lm

CLI:

mlx_lm.generate \
  --model cspenn/Qwen3.6-35B-A3B-MLX-Mixed-4-8 \
  --prompt "Explain the difference between a sparse MoE and a dense transformer." \
  --max-tokens 512

Python:

from mlx_lm import load, generate

model, tokenizer = load("cspenn/Qwen3.6-35B-A3B-MLX-Mixed-4-8")

prompt = "Explain the difference between a sparse MoE and a dense transformer."

# Apply the model's chat template
messages = [{"role": "user", "content": prompt}]
formatted = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)

response = generate(model, tokenizer, prompt=formatted, max_tokens=512, verbose=True)

Thinking mode: Qwen3 supports a /think prefix in the system prompt or user turn to enable extended chain-of-thought reasoning. Omit it for standard fast responses.

Lineage

Qwen/Qwen3.6-35B-A3B  →  unsloth/Qwen3.6-35B-A3B  →  this MLX mixed-4/8 quant

The unsloth base applies fixes for multi-token prediction (MTP) compatibility.

License

Apache 2.0 — see the upstream license.