Qwen3.5-35B-A3B-MXFP4

Rapid42/Qwen3.5-35B-A3B-MXFP4

Qwen3.5 35B A3B — quantized to MXFP4 for Apple Silicon

Converted and optimized by Rapid42 — engineering tools for fast pipelines.

---

What This Is

This is Qwen3.5-35B-A3B (the Mixture-of-Experts variant, ~3.3B active parameters per forward pass) quantized to MXFP4 format using mlx-lm. It runs natively on Apple Silicon via the MLX framework.

The A3B architecture means despite the 35B total parameter count, only ~3.3B parameters activate per token — making this surprisingly fast on-device.

• Total parameters: ~35B
• Active parameters per forward pass: ~3.3B (MoE)
• Quantization: MXFP4 (via mlx-lm 0.31.1)
• Base model: Qwen/Qwen3.5-35B-A3B
• Framework: Apple MLX

---

Hardware Requirements

Tip: MoE models are memory-hungry at load but efficient at inference. The MXFP4 quantization significantly reduces the memory footprint vs. the bf16 original.

---

Quick Start

pip install mlx-lm

from mlx_lm import load, generate

model, tokenizer = load("Rapid42/Qwen3.5-35B-A3B-MXFP4")

messages = [{"role": "user", "content": "Explain mixture-of-experts in plain English."}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_dict=False
)

response = generate(model, tokenizer, prompt=prompt, max_tokens=512, verbose=True)
print(response)

CLI:

mlx_lm.generate \
  --model Rapid42/Qwen3.5-35B-A3B-MXFP4 \
  --prompt "What are the key differences between MoE and dense transformer architectures?" \
  --max-tokens 512

---

Why MXFP4?

MXFP4 (Microscaling FP4) is a quantization format supported natively by Apple's MLX framework. Compared to standard int4:

• Better dynamic range — per-block scaling factors preserve more numerical precision
• Faster on Apple Silicon — MLX is optimised for this format on M-series GPUs
• Smaller than bf16 — roughly 8x reduction in weights size

For a 35B MoE model, this is the practical path to running it locally on an M-series Mac.

---

Conversion

python -m mlx_lm.convert \
  --hf-path Qwen/Qwen3.5-35B-A3B \
  --mlx-path Rapid42/Qwen3.5-35B-A3B-MXFP4 \
  --quantize --q-bits 4 --q-group-size 64

Converted using mlx-lm 0.31.1 on Apple M-series hardware.

---

About Rapid42

Rapid42 builds fast, precise engineering tools — from VFX pipeline utilities to optimized ML model distributions.

→ rapid42.com · ExrToPsd · Level Careers