Qwen3.5-35B-A3B-MLX-VL-4bit-g64

Qwen3.5-35B-A3B MLX VLM Affine W4/g64 With Inline Affine W4 MTP

This is a full vision-language MLX conversion of Qwen/Qwen3.5-35B-A3B, created from exact source revision 59d61f3ce65a6d9863b86d2e96597125219dc754.

Every quantizable target and inline-MTP module uses affine 4-bit weights with group size 64. There are no 8-bit router overrides. As in the standard MLX-VLM conversion scope, the BF16 vision tower and normalization vectors remain unquantized.

The indexed safetensors artifact contains:

• The language target: 512 affine W4/g64 modules, including every MoE router.
• The complete BF16 vision tower: 333 tensors and 446,571,248 parameters.
• The complete native MTP subtree under mtp.*: 13 affine W4/g64 modules, including both MTP routers, plus BF16 norms.

The MTP head reuses the target embedding and LM head as defined by the source architecture.

Runtime requirements

Use mlx-vlm>=0.6.12 for ordinary text and image inference. This artifact was tested with mlx-vlm 0.6.15.

pip install -U "mlx-vlm>=0.6.12" jinja2

mlx_vlm.generate \
  --model EigenLabs/Qwen3.5-35B-A3B-MLX-VL-4bit-g64 \
  --prompt "Explain speculative decoding." \
  --max-tokens 256 \
  --temperature 0

Add --image /path/to/image.jpg for vision-language inference.

Current public mlx-vlm loads and serves the target while ignoring the namespaced mtp.* tensors. This repository does not include a Python Qwen MTP driver. The inline MTP payload is intended for runtimes that implement Qwen3.5 MTP, including Darkbloom's MLX-Swift engine.

Artifact

The MLX converter reports 4.647 effective bits per target weight after scales, biases, and the unquantized vision tower are included.

Machine-readable provenance and verification evidence are in quantization_manifest.json, verification.json, and SHA256SUMS.

Local benchmark

Target-only greedy generation on an Apple M4 Max with mlx-vlm 0.6.15:

• Prompt: 1,701 tokens
• Generation: 128 tokens
• Warm-up: one discarded run
• Measurements: three runs
• Temperature: 0.0

These are local machine measurements, not cross-device guarantees. MTP was not active in this benchmark.

Verification scope

Verified:

• All 27 files in the pinned source snapshot passed Hugging Face cache verification.
• The converted index exactly covers every tensor in all five safetensors files.
• All 525 quantized modules resolve to affine W4/g64; no module has an 8-bit override.
• All 785 source MTP tensors are represented by the 46 serialized MLX tensors.
• Representative affine dequantization checks passed for fusion, attention, expert, and router weights.
• Qwen zero-centered RMSNorm conversion matched exactly.
• chat_template.jinja is byte-identical to the official source.
• A clean mlx-vlm 0.6.15 target-only inference and three-run throughput benchmark completed.

Not yet verified:

• Pixel-grounded image output for this exact conversion.
• Qwen MTP speculative token parity or speedup.
• Video inference.
• Population-level quality after quantization.

License

The base model is released under Apache 2.0. See the original Qwen/Qwen3.5-35B-A3B repository for its model card and license.