Downloads · 30 days
116
12% of all-time downloads
aquaman164/Qwen3.6-35B-A3B-MLX-GPTQ-3.5bpw
Qwen3.6-35B-A3B-MLX-GPTQ-3.5bpw is a text generation model from aquaman164. Use it when you need the model to write or continue text. It is set up for mlx. The card lists the license as apache-2.0.
MLX-native quantization of Qwen/Qwen3.6-35B-A3B (35B-A3B MoE VLM: 40 layers, 256 experts/top-8, hybrid linear-attention) for Apple Silicon. Built on CUDA (2×RTX PRO 6000) with a loss-aware allocation + GPTQ pipeline,…
Downloads · 30 days
116
12% of all-time downloads
All-time downloads
958
Public
Parameters
35.1B
15.3 GB on disk
Likes
1
Public
Click a slice to open those files.
.safetensors15.3 GB · 100%
How the weights are stored.
U3234.7B · 99%
From the Hugging Face model README
MLX-native quantization of Qwen/Qwen3.6-35B-A3B
(35B-A3B MoE VLM: 40 layers, 256 experts/top-8, hybrid linear-attention) for Apple Silicon.
Built on CUDA (2×RTX PRO 6000) with a loss-aware allocation + GPTQ pipeline, packaged in the
standard mlx-lm format (loads with mlx_lm>=0.31).
lm_head 8bit
(inherited from the mlx-community 4bit conversion; lm_head re-quantized to 8bit here).| holdout | KL | PPL |
|---|---|---|
| ja-think | 0.163 | 9.75 → 10.40 (+6.6%) |
| multilingual | 0.108 | 11.02 → 11.78 (+6.9%) |
Plain-RTN mixed-bit at the same size measured +20%/+11% — GPTQ recovers ~⅔ of the damage. (4-bit uniform RTN ≈ +6.6% at ~18 GB; this model matches that quality at 15.5 GB.)
pip install -U mlx-lm # needs >=0.31 (qwen3_5_moe support)
# one-shot
mlx_lm.generate --model /path/to/Qwen3.6-35B-A3B-MLX-GPTQ-3.5bpw \
--prompt "東京の観光名所を3つ教えてください。" --max-tokens 300
# chat REPL
mlx_lm.chat --model /path/to/Qwen3.6-35B-A3B-MLX-GPTQ-3.5bpw
# OpenAI-compatible server
mlx_lm.server --model /path/to/Qwen3.6-35B-A3B-MLX-GPTQ-3.5bpw --port 8080
Notes
Base model © Alibaba Cloud, Apache-2.0 (per Qwen3.6 release). Quantization artifacts derived from the official bf16 checkpoint and the mlx-community 4bit conversion (structure/tokenizer).