Qwen3.6-35B-A3B-mxfp4

<p align="center"> <a href="https://osaurus.ai"><img src="./osaurus-x-banner.png" alt="Osaurus AI"></a> </p> <h3 align="center">Qwen 3.6 35B-A3B &mdash; MXFP4 (MLX)</h3> <p align="center">Open Compute Project MXFP4 quantization of Alibaba's hybrid linear/full attention agentic MoE, with the vision tower preserved.</p> <p align="center"> <a href="https://osaurus.ai"><img src="https://img.shields.io/badge/Web-osaurus.ai-blue" alt="Website"></a>&nbsp; <a href="https://huggingface.co/OsaurusAI"><img src="https://img.shields.io/badge/HF-OsaurusAI-yellow?logo=huggingface" alt="OsaurusAI"></a> </p>

---

Model Details

Quantization details, per tensor category

The quantization map in config.json enumerates all 80 per-layer 8-bit overrides. MXFP4 is the open OCP Microscaling FP4 spec, distinct from NVIDIA's NVFP4 — MLX exposes both as separate --q-mode values; this release uses mxfp4.

---

Architecture notes — what's new vs Qwen 3 / 3.5

• Hybrid attention stack: 30 of 40 layers use Gated DeltaNet, a linear-attention / delta-rule hybrid with a grouped conv1d input path and per-head A_log / dt_bias state — constant memory in sequence length. The other 10 layers (one every 4, given by full_attention_interval: 4) use full softmax attention with attn_output_gate: true (a sigmoid gate on the attention output before o_proj).
• Partial rotary embeddings: only the first 25% of head dim rotates (partial_rotary_factor: 0.25, head_dim 256 → 64 rotated), rope_theta = 1e7. Position metadata for mixed text/image/video (mrope_section: [11, 11, 10], mrope_interleaved: true) is preserved in config.json.
• Mixture of Experts: 256 routed experts, 8 activated per token, moe_intermediate_size: 512, plus a 1 always-active shared expert (same 512 intermediate) gated by sigmoid(shared_expert_gate(x)).
• Router: softmax-topk over expert logits — not DeepSeek-style sigmoid + e_score_correction_bias.

---

Usage

Requires mlx-lm >= 0.30.7 for the qwen3_5_moe text stack and mlx-vlm >= 0.4.4 for image input. This release was packaged with mlx-lm 0.31.2.

Text-only

pip install 'mlx-lm>=0.30.7'

from mlx_lm import load, generate

model, tokenizer = load("OsaurusAI/Qwen3.6-35B-A3B-mxfp4")
print(generate(model, tokenizer,
               prompt="The capital of France is",
               max_tokens=32))

CLI

mlx_lm.generate --model OsaurusAI/Qwen3.6-35B-A3B-mxfp4 \
                --prompt "Write a haiku about Apple Silicon" \
                --max-tokens 100

Reasoning toggle

Qwen 3.6 is a single model with a chat-template-driven thinking switch. Pass enable_thinking as a direct kwarg to apply_chat_template — the chat_template_kwargs dict form only propagates on some tokenizer versions.

msgs = [{"role": "user", "content": "What is 17 × 23?"}]

# Reasoning OFF — inserts a pre-closed <think></think> block
prompt = tokenizer.apply_chat_template(
    msgs, add_generation_prompt=True, enable_thinking=False
)
# Reasoning ON — model fills the <think> block then answers
prompt = tokenizer.apply_chat_template(
    msgs, add_generation_prompt=True, enable_thinking=True
)
print(generate(model, tokenizer, prompt=prompt, max_tokens=256))

Vision (image input)

pip install 'mlx-vlm>=0.4.4'

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

path = "OsaurusAI/Qwen3.6-35B-A3B-mxfp4"
model, processor = load(path)
config = load_config(path)

prompt = apply_chat_template(processor, config,
                             "Describe this image.", num_images=1)
print(generate(model, processor, prompt, "path/to/image.jpg",
               max_tokens=200))

Verified: text, reasoning on/off, single-image VL all produce coherent output on this quant.

Video: the base model supports video inference via transformers, and the bundle preserves video_preprocessor_config.json plus video_token_id / vision_start_token_id / vision_end_token_id. However mlx-vlm 0.4.4 does not expose a video path for qwen3_5_moe — use transformers directly for video.

Audio: the base model is not an Omni variant — no audio path.

---

Hardware notes

19.32 GB weights on disk; once loaded, expect ~19–22 GB resident plus KV cache. Full-attention KV grows with sequence length; linear-attention layers contribute a bounded per-layer SSM state (independent of context).

---

Upstream benchmarks

These are the upstream base-model numbers (Qwen/Qwen3.6-35B-A3B), not evaluations of this MXFP4 quant:

Independent MXFP4 evaluation will be added here as it is produced.

---

Citation

@misc{qwen2026qwen36,
  title  = {Qwen3.6-Plus: Towards Real World Agents},
  author = {Qwen Team},
  year   = {2026},
  url    = {https://qwen.ai/blog?id=qwen3.6}
}

License

Apache 2.0 — inherits from the base model.

---

<p align="center"> Packaged on Apple Silicon with <a href="https://github.com/ml-explore/mlx-lm">mlx-lm</a> 0.31.2.<br> &copy; 2026 Osaurus AI &mdash; <a href="https://osaurus.ai">osaurus.ai</a> </p>