Qwen3.5-9B-M

<div style="display:flex;align-items:center;justify-content:space-between;gap:20px;flex-wrap:wrap"> <div style="flex:1;min-width:260px"> <h1 style="margin:0 0 12px">Mirai's Qwen3.5-9B Medium Quantization</h1> <p style="margin:0">A medium-sized, high-quality quantization of Qwen3.5-9B prepared for efficient local inference on Apple silicon.</p> </div> <img src="https://assets.trymirai.com/images/logo/ml_small_logo.svg" alt="Mirai Labs" width="64" height="64" style="width:80px;height:80px;object-fit:contain;flex:0 0 auto"> </div>

Evaluation data mixture: 45% public agentic, 30% public SFT/long-context, 25% private chat data.

Quickstart

If you are on macOS, the easiest way is to install the mirai Homebrew package and then run the CLI:

brew install mirai
mirai --model trymirai/Qwen3.5-9B-M

Currently only Apple silicon inference is supported. If you want to build things from source, read this overview.

<div style="display:flex;flex-wrap:wrap;gap:12px;align-items:center;margin-top:32px"> <a href="https://trymirai.com/local-models/alibaba-qwen3-5-9b-mirai-mirai-m-4" style="display:inline-flex;align-items:center;justify-content:space-between;width:220px;min-height:53px;padding:0 20px;box-sizing:border-box;border:1px solid #cccccc;border-radius:0;background:#ffffff;color:#3d3d3d;text-decoration:none;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Inter,sans-serif;font-size:15px;font-weight:500;line-height:1.5"> <span>Learn more</span> <span aria-hidden="true" style="font-size:16px;line-height:1">&#8594;</span> </a> </div>

---

Method

Mirai Medium uses 4-bit asymmetric integer quantization with 4-bit zero points, bfloat16 scales, and group size 32. Block-diagonal Random Hadamard Transforms are used to reduce activation and weight outliers. The checkpoint was prepared with post-training quantization followed by quantization-aware distillation.

Citation

If you find our work helpful, feel free to give us a cite.

@misc{mirai-quant,
    title  = {{Mirai Quantization}: Redefining the speed-quality frontier for local LLMs on Apple silicon},
    author = {Artur Chakhvadze and Ryan Mathieu and Roman Knyazhitskiy and Nikolai Voinilenko and Chen-Chen Yeh and Artur Mullakhmetov and Eugene Bokhan and others},
    note   = {In collaboration with others at Mirai Labs},
    month  = {June},
    year   = {2026},
    url    = {https://trymirai.com/blog/quantization}
}

Original model

This is a quantized version of Qwen/Qwen3.5-9B. For architecture details, intended use, evaluations, and limitations, see the original model card.