Ornith-1.5-35B-A3B-oQ2e-mtp

Ornith-1.5-35B-A3B-oQ2e-mtp

This model was quantized using oQ (oMLX v0.6.2) mixed-precision quantization.

Base model: ornith-ai/Ornith-1.5-35B-A3B

Chat template: froggeric/Qwen-Fixed-Chat-Templates (v21.3, original backed up as chat_template.jinja.bak)

Quantization details

• Model type: qwen3_5_moe
• Bits: 2
• Group size: 64
• Mode: affine
• Format: MLX safetensors
• MTP: Preserved (mtp_num_hidden_layers: 1 — built-in MTP head, no external donor)
• Calibration: oQ2e (enhanced, imatrix-based)
• Note: text-only variant (vision tower dropped, −0.89 GB) to keep the MTP head intact

Environment

• Hardware: M5 MacBook Air 32GB
• Inference Framework: oMLX v0.6.2
• Max Concurrent Requests: 4
• Settings:
  • Thinking: Disabled
  • TurboQuant KV Cache: Enabled (4-bit)
  • Lightning MTP: Disabled (recommended — see note below)

⚠️ Lightning MTP

中文：本模型建议不要开启 Lightning MTP。MoE 架构模型 MTP 效果有差异，请先自行测试再决定是否开启。本卡的性能与智能基准数据均在不开启 Lightning MTP 的情况下测得。

English: For this model it is recommended to leave Lightning MTP disabled. MTP effectiveness varies across MoE-architecture models — test on your own setup before deciding whether to enable it. All benchmark results on this card were measured with Lightning MTP off.

Performance Benchmarks

Note: Results are for reference only and may vary depending on hardware, software configuration, and workload.

Single Request Results

Continuous Batching (pp1024 / tg128)

Intelligence Benchmark

Note: Each benchmark round tests only 30 questions. Results are for reference only.