Qwen3.6-27B-NVFP4-MLX

KCh3dRi4n/Qwen3.6-27B-NVFP4-MLX

Qwen3.6-27B-NVFP4-MLX is a image-text-to-text model from KCh3dRi4n. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for mlx. The card lists the license as other.

MLX conversion of nvidia/Qwen3.6-27B-NVFP4 (NVIDIA ModelOpt quantization of Qwen3.6-27B) for Apple Silicon, preserving NVIDIA's exact mixed-precision scheme.

Downloads · 30 days

5% of all-time downloads

All-time downloads

304

Public

Parameters

26.9B

39.1 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors24 GB · 100%

Parameter types

How the weights are stored.

U3224.3B · 88%

Try a prompt

Base models

nvidia/Qwen3.6-27B-NVFP4

Model card

From the Hugging Face model README

Qwen3.6-27B-NVFP4-MLX

MLX conversion of nvidia/Qwen3.6-27B-NVFP4 (NVIDIA ModelOpt quantization of Qwen3.6-27B) for Apple Silicon, preserving NVIDIA's exact mixed-precision scheme.

≈ 6.60 bits / weight, 21 GB on disk.
MLP projections → NVFP4 (mode: "nvfp4", group_size: 16, 4-bit).
Attention projections → MXFP8 (mode: "mxfp8", group_size: 32, 8-bit) — NVIDIA kept these at FP8; re-expressed as MLX mxfp8.
embed_tokens, lm_head, in_proj_a/b, all norms, MTP head, vision tower → BF16 (native), exactly as NVIDIA left them.

This is the NVIDIA-faithful build: every layer keeps the precision NVIDIA assigned it, so accuracy tracks NVIDIA's published NVFP4 checkpoint. (A uniform 4-bit "all-NVFP4" variant would be ~16.9 GB but quantizes attention and the embedding/output head more aggressively.)

Precision map

layers	precision	bits
MLP `gate/up/down_proj` (64)	NVFP4	4
Attention `q/k/v/o_proj`, `in_proj_qkv/z`, `out_proj` (208)	MXFP8	8
`embed_tokens`, `lm_head`, `in_proj_a/b`, norms	BF16	16
MTP head (`mtp.*`, 15 tensors)	BF16	16
vision tower (`vision_tower.*`, 333 tensors)	BF16	16

How it was converted

Dequantize NVIDIA's ModelOpt tensors to BF16 (unpack_fp4 · weight_scale · weight_scale_2 for NVFP4; weight · weight_scale for FP8), validated against compressed-tensors.
Re-quantize per layer with mlx_lm — MLP → nvfp4, attention → mxfp8, everything NVIDIA left in BF16 stays BF16 (plus the Qwen3.5 RMSNorm/conv1d sanitization).
Graft back the BF16 MTP (mtp.*) and vision tower (vision_tower.*, model-vision.safetensors) and restore vision_config.

Files

file	contents	dtype
`model-0000{1..5}-of-00005.safetensors`	language model	NVFP4 (MLP) + MXFP8 (attn) + BF16
`model-mtp.safetensors`	MTP head (15 tensors)	BF16
`model-vision.safetensors`	vision tower (333 tensors)	BF16

Usage

Built for oMLX (Native MTP / VLM). Text-only generation also works with mlx-lm:

mlx_lm.generate --model KCh3dRi4n/Qwen3.6-27B-NVFP4-MLX \
  --prompt "Explain NVFP4 quantization." --max-tokens 128

For stock (unpatched) mlx-lm / mlx-vlm, remove model-mtp.safetensors before loading text-only: those loaders shift RMSNorm weights when mtp.* tensors are present, which would double-shift this already-converted model. oMLX handles the MTP-preserved layout directly.

Attribution

Base weights & quantization: NVIDIA — nvidia/Qwen3.6-27B-NVFP4 (NVIDIA Open Model License).
Original model: Qwen — Qwen/Qwen3.6-27B.

Task: Image-Text-to-Text
Library: mlx
Type: qwen3_5
License: other
Created: Jul 9, 2026
Updated: Jul 9, 2026