Downloads · 30 days
38
11% of all-time downloads
EigenLabs/Qwen3.6-35B-A3B-MLX-VL-4bit-g64-router8
Qwen3.6-35B-A3B-MLX-VL-4bit-g64-router8 is a image-text-to-text model from EigenLabs. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for mlx. The card lists the license as apache-2.0.
This is a full vision-language MLX conversion of Qwen/Qwen3.6-35B-A3B, created from exact source revision 995ad96eacd98c81ed38be0c5b274b04031597b0.
Downloads · 30 days
38
11% of all-time downloads
All-time downloads
340
Public
Parameters
35.4B
41.7 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors21.3 GB · 100%
How the weights are stored.
U3234.9B · 99%
From the Hugging Face model README
This is a full vision-language MLX conversion of
Qwen/Qwen3.6-35B-A3B,
created from exact source revision
995ad96eacd98c81ed38be0c5b274b04031597b0.
The single indexed safetensors weight set contains all three components:
mtp.*: 835,715,072 matrix parameters in MXFP8/g32 and 8,925,696 sensitive fusion/router/gate/norm parameters in BF16.There is no mtp/ directory, separate MTP repository, or mtp.safetensors.
The MTP head reuses the target embedding and LM head, as intended by the source
architecture.
Use mlx-vlm>=0.6.12 for normal text and image inference. Earlier Qwen3.5-MoE
VLM loaders could reapply the source RMSNorm conversion to serialized MLX
weights.
pip install -U "mlx-vlm>=0.6.12"
Normal mlx_vlm.load() ignores the namespaced mtp.* tensors safely. The
included mtp_exact_generate.py constructs the drafter in memory directly from
those entries in the same index. It does not use --draft-model, create a
temporary weight copy, or download another repository.
Stock MLXVLM 0.6.12 uses a batched target-verification linear path whose greedy arithmetic can differ from ordinary one-token decoding for this mixed W4/W8 MoE target. Exact output parity requires target-verification linear projections to run one time position at a time. The artifact verification used that minimal runtime correction while leaving MoE expert gathers batched. Without it, a rejected proposal can change later output; do not claim exact speculative decoding from unpatched 0.6.12.
| Property | Value |
|---|---|
| Indexed tensors | 2,120 |
| Pre-existing VLM tensors | 2,090 |
| Serialized MTP tensors | 30 |
| Source MTP parameters | 844,640,768 |
| MXFP8/g32 MTP parameters | 835,715,072 |
| BF16 MTP parameters | 8,925,696 |
| MTP payload | 879,682,560 bytes (0.819268 GiB) |
| Combined tensor payload | 21,281,612,512 bytes (19.820046 GiB) |
| Safetensors shards | 4 |
The machine-readable provenance, coverage, validations, and hashes are in
quantization_manifest.json, verification.json, and SHA256SUMS.
python -m mlx_vlm generate \
--model EigenLabs/Qwen3.6-35B-A3B-MLX-VL-4bit-g64-router8 \
--image /path/to/image.jpg \
--prompt "Describe this image." \
--max-tokens 256 \
--temperature 0 \
--thinking-mode disabled
Strict eager loading from the combined artifact passed:
ORCHID-7319, count 27, target CYAN TRIANGLE, and the correct ordered shapes/colors.{"sum":423,"check":"ok"}.All ordinary outputs match the pre-MTP VLM artifact because every original VLM tensor is value-identical.
python mtp_exact_generate.py \
--model EigenLabs/Qwen3.6-35B-A3B-MLX-VL-4bit-g64-router8 \
--draft-block-size 3 \
--prompt "Explain speculative decoding." \
--max-tokens 256 \
--temperature 0
The final parity suite used five workloads, three target-only runs and three MTP runs per workload, for 30 paired runs total. It included structured text, technical prose, code, a length-limited sequence, and image prefill.
| Metric | Result |
|---|---|
| Exact token parity | 15/15 MTP runs |
| Exact text parity | 15/15 MTP runs |
| Drafted tokens | 1,671 |
| Accepted draft tokens | 1,353 |
| Rejected draft tokens | 318 |
| Aggregate acceptance | 80.97% |
| Median per-case speedup | 1.284x |
| Per-case speedup range | 1.112x to 1.518x |
Every case was faster than target-only decoding. The 318 rejections exercised target cache rollback repeatedly, including three rejections after image prefill. These are local M4 Max measurements, not cross-device guarantees.
fc, routers, gates, and norms remain BF16.Image and text inference plus greedy MTP parity are verified. Video processor files are preserved, but video inference was not tested. The validation suite is targeted evidence, not a broad multimodal or population-quality benchmark.
The base model is released under Apache 2.0. See the original
Qwen/Qwen3.6-35B-A3B
repository for its model card and license.