Qwen3.6-27B-MXFP4-mlx

Qwen3.6-27B — Unsloth MXFP4 + MXFP8 (MLX)

Brooooooklyn/Qwen3.6-27B-MXFP4-mlx is an Apple Silicon MLX quantization of Qwen/Qwen3.6-27B. The model has a 64-layer dense Qwen3.5-family language backbone with 48 linear-attention and 16 full-attention layers, a BF16 vision tower, and one MTP layer.

This model is part of the Qwen Unsloth tensor-class recipe for MLX on macOS and DGX collection.

The source was the BF16 checkpoint at revision 1b559cf7215ebe67ff10758e14f6293ba883223b.

Update — re-converted 2026-09-01

The MXFP4 and MXFP8 weights are re-encoded. mlx-node now picks each MX block's shared E8M0 exponent itself instead of rounding log2(amax) to nearest: MXFP4 tries both candidate exponents and keeps the lower squared error, and MXFP8 takes the ceiling so a block can never saturate. Measured MXFP8 weight error across a checkpoint drops from 6.91% to 2.66%.

Weight bytes changed; the tensor-class map, shard layout and tensor counts are unchanged. Earlier revisions remain in this repository's history.

Quantization recipe

This is a data-free, weight-only Apple translation of the Unsloth Qwen3.6 NVFP4 recipe:

• tensors assigned NVFP4 by the recipe are stored as MXFP4, 4-bit with group size 32;
• tensors assigned FP8 by the recipe are stored as MXFP8, 8-bit with group size 32;
• every excluded tensor stays BF16.

This ports the recipe's tensor-class selection. It does not claim numerical parity with NVIDIA NVFP4/FP8 activation quantization. Both MXFP classes use MLX weight-only quantized matmul with BF16/A16 activations. The artifact contains no activation-calibration or FP8 KV-cache-calibration data, global scales, or W4A4/W8A8 execution metadata. No imatrix, calibration dataset, or AWQ-style pre-scaling was used, and no upstream numerical or performance parity is claimed.

The resulting config contains 168 MXFP4 module entries and 233 MXFP8 module entries. The last eight dense FFNs intentionally use the higher-precision MXFP8 class.

Compatibility and usage

This checkpoint requires @mlx-node/lm and @mlx-node/core 0.0.8 or newer. The published prebuilt macOS binary requires macOS 26 or newer; macOS 14 or newer can use a source build.

npm install @mlx-node/lm@^0.0.8 @mlx-node/core@^0.0.8

import { loadSession } from '@mlx-node/lm';

const session = await loadSession('./Qwen3.6-27B-MXFP4-mlx');
const result = await session.send('Explain the purpose of a unit test in one sentence.');
console.log(result.text);

Reproduction

Converter: mlx-node at or after PR #131, which made the tuned MX weight encoders and the NVFP4 power-of-two lift unconditional. v0.0.8 reproduces the earlier revision of this repository, not the current weights. The normalized reproducible invocation from the mlx-node repository root was:

mlx convert \
  --input .cache/models/qwen3.6-27b \
  --output .cache/models/qwen3.6-27b-unsloth-mxfp4-mlx \
  --model-type qwen3_5 \
  --dtype bfloat16 \
  --quantize \
  --q-recipe unsloth \
  --q-mxfp

Validation

A local mlx-node load-and-generate smoke test completed successfully and produced the one-token output OK. This functional smoke test is separate from the performance benchmark below and does not validate model quality. All 15 mtp.* tensor entries remain BF16 without quantization sidecars.

Benchmark

Measured on an Apple M5 Max with 128 GiB of unified memory, Darwin 25.5.0/arm64, Node.js 24.13.1, and @mlx-node/lm, @mlx-node/core, and @mlx-node/core-darwin-arm64 0.0.8.

The table reports the median of three measured runs. Each run used a fresh child process and fresh model load, with zero warmups and a 60-second cooldown. Generation was deterministic: temperature: 0, reasoning effort none, 106 prompt tokens, and a 512-token generation limit. All three runs generated 512 tokens and ended with finishReason: "length".

Raw per-run samples and the machine-readable summary are available in benchmark.json.

The model files were read from external storage. Load timing is sensitive to external-storage and OS file-cache state; the three measured loads ranged from 59.073 to 283.375 seconds. Treat load time as host- and cache-specific. These measurements describe only this prompt and configuration and must not be used to infer model quality.

License and attribution

The source model card declares the Apache-2.0 license. Model capability and training credit belong to the Qwen Team. The tensor-class recipe is credited to Unsloth, and this repository only converts the BF16 source weights to an MLX-native mixed MXFP4/MXFP8 representation.