Downloads · 30 days
272
5% of all-time downloads
Brooooooklyn/Qwen3.6-35B-A3B-UD-Q2_K_XL-mlx
Qwen3.6-35B-A3B-UD-Q2_K_XL-mlx is a text generation model from Brooooooklyn. Use it when you need the model to write or continue text. It is set up for mlx-node. The card lists the license as apache-2.0.
2-bit base mixed-precision quantization of Qwen/Qwen3.6-35B-A3B for Apple Silicon, using the Unsloth Dynamic quantization strategy via mlx-node.
Downloads · 30 days
272
5% of all-time downloads
All-time downloads
5.3K
Public
Parameters
35.1B
14.2 GB on disk
Likes
7
Public
Click a slice to open those files.
.safetensors14.2 GB · 100%
How the weights are stored.
U3234.3B · 98%
From the Hugging Face model README
2-bit base mixed-precision quantization of Qwen/Qwen3.6-35B-A3B for Apple Silicon, using the Unsloth Dynamic quantization strategy via mlx-node.
| Original (BF16) | This Model | |
|---|---|---|
| Size | ~66 GB | 14 GB |
| Format | SafeTensors (sharded) | SafeTensors (sharded) |
| Precision | BF16 uniform | Mixed 2/…/8-bit + BF16 |
| Repo | GGUF Equivalent | Size | Decode (tok/s) |
|---|---|---|---|
| Brooooooklyn/Qwen3.6-35B-A3B-UD-Q2_K_XL-mlx (this model) | UD-Q2_K_XL | 14 GB | 63.7 |
| Brooooooklyn/Qwen3.6-35B-A3B-UD-Q3_K_XL-mlx | UD-Q3_K_XL | 18 GB | 59.2 |
| Brooooooklyn/Qwen3.6-35B-A3B-UD-MXFP4_K_XL-mlx | — | 21 GB | 54.4 |
| Brooooooklyn/Qwen3.6-35B-A3B-UD-NVFP4_K_XL-mlx | — | 22 GB | 59.1 |
| Brooooooklyn/Qwen3.6-35B-A3B-UD-Q4_K_XL-mlx | UD-Q4_K_XL | 22 GB | 55.1 |
| Brooooooklyn/Qwen3.6-35B-A3B-UD-Q5_K_XL-mlx | UD-Q5_K_XL | 26 GB | 51.8 |
| Brooooooklyn/Qwen3.6-35B-A3B-UD-Q6_K_XL-mlx | UD-Q6_K_XL | 31 GB | 54.4 |
| Brooooooklyn/Qwen3.6-35B-A3B-UD-MXFP8_K_XL-mlx | — | 35 GB | 47.6 |
| Brooooooklyn/Qwen3.6-35B-A3B-UD-Q8_K_XL-mlx | UD-Q8_K_XL | 36 GB | 45.9 |
Benchmarked on Apple M3 Max 128GB via examples/lm.ts (best decode tok/s across turns 2–4, steady-state).
Steady-state decode: 63.7 tok/s on Apple M3 Max 128GB (best of turns 2–4, examples/lm.ts capitals chat with reasoningEffort: 'low').
Decode is memory-bandwidth bound on Apple Silicon — fewer bytes per token directly translates to higher throughput. The MoE architecture activates only 8 of 256 experts per token (~3B active out of 35.9B total), and the compiled C++ forward graph fuses the per-layer dispatch.
| Weight | Bits | Rationale |
|---|---|---|
embed_tokens | 4-bit | KLD ~0.15 — very low sensitivity |
lm_head | 5-bit | KLD ~0.05 — safest tensor |
self_attn.q/k/v_proj | 4-bit + AWQ | KLD ~1.5–2.9, AWQ via layernorm |
linear_attn.in_proj_qkv/z | 4-bit + AWQ | KLD ~2.9, AWQ via layernorm |
self_attn.o_proj | bf16 | NOT AWQ-correctable |
linear_attn.out_proj | bf16 | KLD ~6.0 — worst tensor |
down_proj | 3-bit | "Slightly more sensitive" |
gate_proj, up_proj | 2-bit | base bits |
| Router gates | 8-bit | MoE routing accuracy |
| GDN params (A_log, etc) | bf16 | State-space dynamics |
Based on Unsloth Dynamic 2.0 per-tensor KLD analysis. Sensitive layers get higher bits with AWQ correction, while the bulk of FFN expert weights are aggressively quantized. imatrix AWQ pre-scaling amplifies important weight channels and fuses inverse scales into preceding layer norms (zero inference overhead).
AWQ-correctable projections (q/k/v, in_proj_qkv/z) are quantized at 4-bit via input_layernorm. Non-AWQ-correctable projections (o_proj, out_proj) are kept at bf16 — their inputs come from attention/GDN computation, not from a norm layer.
| Parameter | Value |
|---|---|
| Total parameters | 35.9B (3B active per token) |
| Hidden size | 2,048 |
| Layers | 40 (30 linear + 10 full attention) |
| Attention heads | 16 (2 KV heads, GQA 8:1) |
| Head dimension | 256 |
| Experts | 256 per MoE layer, top-8 routing |
| Vocab size | 248,320 |
| Max context | 262,144 tokens |
import { loadSession } from '@mlx-node/lm';
const session = await loadSession('./Qwen3.6-35B-A3B-UD-Q2_K_XL-mlx');
for await (const event of session.sendStream('Explain the hybrid attention mechanism in Qwen3.6.', {
config: { maxNewTokens: 2048, temperature: 0.6, reasoningEffort: 'low' },
})) {
if (!event.done) process.stdout.write(event.text);
}
mlx convert \
-i Qwen3.6-35B-A3B \
-o Qwen3.6-35B-A3B-UD-Q2_K_XL-mlx \
-q --q-bits 2 --q-recipe unsloth \
--imatrix-path imatrix_unsloth.gguf
Apache 2.0 (inherited from base model).