GRM-3.2-Sky-6bit-XL-mlx

leonsarmiento/GRM-3.2-Sky-6bit-XL-mlx

GRM-3.2-Sky by OrionLLM — a 35B-A3B sparse MoE fine-tuned from Ornith-1.0-35B for long-horizon agentic tasks and extremely difficult reasoning, quantized for Apple Silicon using the BaseQuant_XL 6/8-bit recipe.

Only ~3B parameters active per token — the decode speed of a small model with the capacity of a 35B one. GRM-3.2-Sky is OrionLLM's flagship: it targets failure modes like drift and loss of goal state over extended multi-step workflows, and posts 81.4 SWE-bench Verified, 87.7 LiveCodeBench v6, and 96.3 AIME26.

This is a full multimodal build — the vision tower is preserved.

About XL Quantization

BaseQuant_XL is a fully data-agnostic, static quantization. No calibration dataset, no sensitivity analysis, no importance matrix. Precision is allocated purely by architectural role — routing-critical layers get higher precision, bulk expert parameters get lower precision. The result is a transparent, faithful capture of the source model.

Data-dependent calibration quantizations (iMatrix, AWQ, GPTQ, oQ, oQ4e, etc.) use a calibration set to guide bit allocation. This can produce a skewed representation of the model: domains well-represented in the calibration data (English, popular topics, public or leaked benchmarks) are preserved better, while underrepresented domains (non-English languages, niche use cases, your own data) are preserved worse. XL avoids this trade-off entirely — it generalizes honestly because it is never fit to any particular data distribution.

Quickstart

pip install -U mlx-vlm

python -m mlx_vlm.generate --model leonsarmiento/GRM-3.2-Sky-6bit-XL-mlx --max-tokens 256 --temperature 0.6 --top-p 0.95 --prompt "Implement an LRU cache in Python with O(1) get/put."

Works with LM Studio — vision mmproj included. Thinking mode is on by default (emits <think>...</think>).

Quantization Strategy

BaseQuant_XL recipe — precision is allocated by layer importance, not applied uniformly:

• Bits per weight: ~6.8 · Total size: ~30 GB · Group size: 64

Notes specific to this build:

• Source config declares an MTP (multi-token prediction) layer but ships no MTP weights — the phantom declaration is stripped in this build so engines don't mis-flag it.
• Like its Ornith-1.0 parent, the source stores experts individually (256 per layer) — converted via a custom sanitize that merges them into stacked expert tensors.
• Ships the qwen3.6-froggeric-v21.3 chat template — thinking on by default, preserve_thinking, <|think_on|>/<|think_off|> inline toggles, XML and JSON tool-call formats.

Recommended Inference Parameters

Thinking is on by default (<think>...</think> before the answer); with a reasoning parser enabled the chain-of-thought is returned in a separate reasoning_content field.

Model Overview

Source Model Benchmarks (from OrionLLM)

Source

OrionLLM/GRM-3.2-Sky