Qwen3.5-27B-MXFP4

Rapid42/Qwen3.5-27B-MXFP4

Qwen3.5 27B — quantized to MXFP4 for Apple Silicon

Converted and optimized by Rapid42 — engineering tools for fast pipelines.

---

What This Is

This is Qwen3.5-27B (dense architecture) quantized to MXFP4 format using mlx-lm. Runs natively on Apple Silicon via the MLX framework.

The 27B dense model hits a sweet spot: larger reasoning capacity than the 7B/14B range, but more practical than 72B for local deployment on M-series hardware.

• Parameters: ~27B (dense)
• Quantization: MXFP4 (via mlx-lm 0.31.1)
• Base model: Qwen/Qwen3.5-27B
• Framework: Apple MLX

---

Hardware Requirements

---

Quick Start

pip install mlx-lm

from mlx_lm import load, generate

model, tokenizer = load("Rapid42/Qwen3.5-27B-MXFP4")

messages = [{"role": "user", "content": "Write a Python function to parse EXR layer names."}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_dict=False
)

response = generate(model, tokenizer, prompt=prompt, max_tokens=512, verbose=True)
print(response)

CLI:

mlx_lm.generate \
  --model Rapid42/Qwen3.5-27B-MXFP4 \
  --prompt "Summarize the key advantages of Apple Silicon for local LLM inference." \
  --max-tokens 512

---

Why This Model?

Qwen3.5-27B is a strong general-purpose model that punches above its weight class on coding, reasoning, and multilingual tasks. At MXFP4 it loads in ~15GB unified memory — making it the best quality-per-GB option in the Qwen3.5 dense lineup for M-series Macs.

Compared to the 35B MoE (A3B):

• Simpler architecture — no routing overhead
• More predictable latency — consistent tokens/sec
• Better for single-GPU batch — no expert load balancing

---

Why MXFP4?

MXFP4 (Microscaling FP4) uses per-block scaling factors, preserving more dynamic range than standard int4. MLX handles this natively on Apple Silicon — no plugins or special runtimes needed.

---

Conversion

python -m mlx_lm.convert \
  --hf-path Qwen/Qwen3.5-27B \
  --mlx-path Rapid42/Qwen3.5-27B-MXFP4 \
  --quantize --q-bits 4 --q-group-size 64

Converted using mlx-lm 0.31.1.

---

About Rapid42

Rapid42 builds fast, precise engineering tools — from VFX pipeline utilities to optimized ML model distributions.

→ rapid42.com · ExrToPsd · Level Careers