Qwen3.5-27B-MLX-4bit

Qwen3.5-27B MLX 4-bit

This is an unofficial MLX 4-bit quantization of Qwen/Qwen3.5-27B.

It was converted from the official Hugging Face weights with mlx-lm. This repository is not affiliated with or endorsed by Qwen or Alibaba.

Conversion

mlx_lm.convert \
  --hf-path Qwen/Qwen3.5-27B \
  --mlx-path Qwen3.5-27B-MLX-4bit \
  --quantize \
  --q-bits 4 \
  --q-group-size 64

Quantization reported by mlx_lm.convert:

Quantized model with 4.501 bits per weight.

Local Usage

mlx_lm.generate \
  --model Qwen3.5-27B-MLX-4bit \
  --prompt "用一句话说明你是什么模型。" \
  --max-tokens 80 \
  --chat-template-config '{"enable_thinking": false}'

OpenAI-compatible local server:

mlx_lm.server \
  --model Qwen3.5-27B-MLX-4bit \
  --host 127.0.0.1 \
  --port 8080 \
  --chat-template-args '{"enable_thinking": false}'

Smoke Test

Tested on Apple Silicon with mlx-lm 0.31.3:

Prompt: 19 tokens, 41.697 tokens-per-sec
Generation: 21 tokens, 15.582 tokens-per-sec
Peak memory: 15.411 GB

Only text generation has been smoke-tested. The base model is multimodal, but this MLX conversion has not yet been validated for image input.

License

The original model is released under Apache-2.0. See the upstream model page and license:

• https://huggingface.co/Qwen/Qwen3.5-27B
• https://huggingface.co/Qwen/Qwen3.5-27B/blob/main/LICENSE