Qwen3.6-27B-oQ4-fp16-mtp

Qwen3.6 oQ4 Text-Only with Native MTP (Optimized for Apple Silicon)

This repository contains custom-quantized, text-only configurations of the Qwen3.6 model family, optimized explicitly for local repository-level agentic coding on Apple Silicon using the oMLX inference engine.

🎯 Why This Was Created

Standard open-weight releases and community pre-converted MLX files (like standard Unsloth) routinely strip out Multi-Token Prediction (mtp.*) tensors during conversion. While this saves minimal disk space, it completely disables the hardware-level Native MTP decoding pipeline on Apple Silicon.

This project was built to solve specific long-context constraints encountered on Apple Silicon:

• The Dense Speed Bottleneck: Dense architectures like the Qwen3.6-27B provide significantly higher coding accuracy, sharper reasoning, and far fewer repetitive loops than sparse MoE setups. However, they naturally run much slower. Preserving the original MTP heads unlocks a ~1.5x generation speedup to make dense coding completely viable locally.
• Overcoming the 128k Context Wall: Traditional backends like LM Studio frequently experience severe latency degradation or choke entirely when context windows approach 128k tokens due to unoptimized KV cache regeneration. Moving to oMLX's specialized two-tier caching eliminates this overhead.
• Maximizing Efficiency on Lighter Hardware: While oQ8 requires a high-end machine (like 64GB/96GB RAM), oQ4 (4-bit) is optimized to run efficiently on mainstream Macs with 24GB or 32GB/36GB Unified Memory, leaving plenty of headroom for system tasks and active development workspaces.

---

🚀 Key Differences

---

💻 Hardware & RAM Recommendations

---

🛠️ Quantization Settings

This model was quantized using oMLX v0.4.4 with the following settings:

• Source Model: Qwen/Qwen3.6-27B
• Sensitivity Model: None (use source model)
• oQ Level: oQ4
• Text Only: Enabled (Excludes vision encoder weights, saving ~2-3% memory footprint)
• Preserve MTP weights: Enabled (Preserves mtp.* tensors for Native MTP support)
• Non-quant weight dtype: float16 (Provides ~20% faster prefill on M1/M2 Apple Silicon)

---

⚙️ Optimized oMLX Settings (v0.4.4)

To replicate the ultra-long context development workspace used with OpenCode 1.16.2, apply the following server specifications in your oMLX dashboard:

Model Basic Settings

• Reasoning Parser: qwen_3_coder (Isolates CoT thoughts to protect IDE syntax)
• CTX Window: 262,144
• Max Tokens: 32,768
• Temperature: 0.6
• Top P / Top K: 0.95 / 20
• Min P: 0
• Repetition / Presence Penalty: 1 / 0

Model Advanced Settings

• Enabled Thinking: Checked (True)
• Chat Template Kwargs: enable_thinking: true, preserve_thinking: true
• Native MTP: Checked (True)

Resource Management & Cache

• Memory Guard: Aggressive (Enforces strict macOS memory/swap cleanup)
• Hot Cache Limit (RAM): 20GB (Allocated for high-speed Unified Memory history)
• Cold Cache Limit (SSD): 371GB (Serialized safetensors storage for context overflow)
• Max Concurrent Requests: 2 (Prevents splitting the 400 GB/s bandwidth bus)
• Embedding Batch Size: 32
• Chunked Prefill: Enabled (Prevents instantaneous out-of-memory crashes on large files)
• Burst Decode: Aggressive (Coalesces tokens for maximum raw typing speed)
• Initial Cache Blocks: 256
• SSE Keepalive Mode: Chunk

---

🌡️ Thermal Optimization Notice

Sustained local execution across deep context windows will heavily tax the Apple Silicon GPU/CPU, leading to internal heat buildup. Because Apple's default fan profiles prioritize silence over longevity, they delay high-RPM cooling until thermal throttling is already occurring.

To manage this proactively during extended coding agent sessions, use smc-command—an adapted CLI fan utility tailored to trigger more aggressive fan curves based on exact thermal sensors.

• Source Repository: smc-command on Codeberg