GRM-3.2-Sky-abliterated-FP8

Crata/GRM-3.2-Sky-abliterated-FP8

GRM-3.2-Sky-abliterated-FP8 is a image-text-to-text model from Crata. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.

本版本说明（GRM-3.2-Sky-abliterated-FP8）：这是 GRM-3.2-Sky-abliterated（abliterated bf16）的 FP8 量化版。语言塔采用 FP8DYNAMIC（动态激活缩放，无需校准数据），视觉塔 / lmhead / MoE 路由门控 / 共享专家门控保持 BF16 原样。config 为嵌套结构（textconfig 含 YaRN 1M 上下文扩展），quantization…

Downloads · 30 days

15% of all-time downloads

All-time downloads

Public

Parameters

34.7B

36.7 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors36.7 GB · 100%

Parameter types

How the weights are stored.

F8_E4M333.6B · 96%

Try a prompt

Base models

Crata/GRM-3.2-Sky-abliterated

Model card

From the Hugging Face model README

RUN

VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 VLLM_USE_FASTOKENS=1 CUDA_VISIBLE_DEVICES=0 CUDA_DEVICE_ORDER=PCI_BUS_ID vllm serve Crata/GRM-3.2-Sky-abliterated-FP8 \
  --gpu-memory-utilization 0.9 \
  --kv-cache-dtype fp8 \
  --max-model-len 1010000 \
  --max-num-seqs 8 \
  --max-num-batched-tokens 16384 \
  --enable-chunked-prefill \
  --enable-prefix-caching \
  --async-scheduling \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder \
  --reasoning-parser qwen3 \
  --served-model-name GRM-3.2-Sky-abliterated-FP8 \
  --host 0.0.0.0 --port 8000

本版本说明（GRM-3.2-Sky-abliterated-FP8）：这是 GRM-3.2-Sky-abliterated（abliterated bf16）的 FP8 量化版。语言塔采用 FP8_DYNAMIC（动态激活缩放，无需校准数据），视觉塔 / lm_head / MoE 路由门控 / 共享专家门控保持 BF16 原样。config 为嵌套结构（text_config 含 YaRN 1M 上下文扩展），quantization_config.ignore 已按视觉正则声明排除视觉塔——vLLM 可直接按多模态加载。

量化说明

项目	说明
量化对象	仅语言塔 Linear（`targets="Linear"`），31030 个权重张量 → FP8（E4M3）
保留 BF16	视觉塔全部 333 张量、`lm_head`、`mlp.gate`（路由）、`shared_expert_gate`
方案	`FP8_DYNAMIC`（llm-compressor / compressed-tensors），动态 per-tensor 激活缩放
上下文	config 内置 YaRN（factor 4.0，原始 262K → 1M），配合 `VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 --max-model-len 1010000`
显存	约 35G（vs bf16 66G），单张 48G 卡可载
消融说明	消融方法/指标/已知局限与 bf16 版完全一致，见下文（本文件其余部分与 bf16 版 README 相同）

本版本说明（GRM-3.2-Sky-abliterated）：这是在 GRM-3.2-Sky 基础上做 abliteration（权重消融） 得到的破限版本，其余能力与原版一致。

破限参数速览

项目	说明
破限方法	Abliterix 1.12.2 方向消融：基于激活对比提取拒绝方向，正交投影移除 + 逐分量 LoRA 引导 + MoE 专家抑制
引导分量	注意力 `q/k/v/o_proj` + `mlp.down_proj`（各带高斯强度分布），MoE 专家抑制 `n_suppress=7, router_bias=-0.50, expert_ablation_weight=6.70`
调优要点	正交投影 + winsorize 向量；评测在思考域（`enable_thinking=True`）下进行，与推理域对齐
指标（评测 100 项）	拒答 91 → 12（基线 91/100 → 消融后 12/100），KL 散度 0.011
随机种子	`2117268343`
轮次	20 轮 / 5 warmup，胜出轮 trial index = 14
复现	见 `reproduce/reproduce.json`（含完整参数、哈希、环境清单，`reproducible` 标记）

已知局限与失败尝试记录

本版本为四轮迭代（V2/V3/V4-80/本版）中实测行为最稳的一版，但需了解以下事实：

概率性冷启动退化：消融对身份/自我认知与高敏感话题的生成通路有残余损伤。当此类问题作为会话第一句出现时，有一定概率（实测约 1/3）触发无限复读（如整段感叹号或短语循环）；一旦上下文中已有正常问答（热启动），后续同类问题表现正常。这是该方法在此底座上的固有缺陷，四个版本的评测指标均无法预测它。
80 轮搜索的反直觉结果：延长搜索（20→80 轮）在评测指标上更优（拒答 12→7），但服务端实测更差——身份问题冷启动直接进入循环。教训：更低的评测拒答率不等于更好的实际行为；评测生成上限（100 token）内正文可能尚未开始，指标与真实体验脱节。
评测裁判的"死循环"盲区：基于关键词的判官无法区分"正常作答"与"复读循环"（循环文本不含拒答关键词，会被记为合规），且短生成上限会让循环藏在分数之外。我们后续试验了引擎内自裁判（restore 原版权重对每条输出做 R/C 三态分类、垃圾输出 3 倍罚分），管线可行，但 20 轮验证显示 warmup 期垃圾率高达 70~91%，TPE 尚未起效即终止——该方法在此底座上是否存在"零退化"配方仍未被证明。
使用建议：多轮对话场景基本可用；若首问触发复读，重发一次或先说一句日常话即可恢复。对稳定性要求高的场景请使用原版 GRM-3.2-Sky。

⚠️ 消融会削弱安全对齐，可能生成对敏感/危险指令的响应；本模型仅用于研究评估，请勿用于非法用途。余下说明与原版一致。

Quantization Notes (English)

GRM-3.2-Sky-abliterated-FP8 is the FP8 build of GRM-3.2-Sky-abliterated. The language tower is quantized with FP8_DYNAMIC (dynamic per-tensor activation scales, no calibration data needed — 31,030 weight tensors → E4M3); the vision tower (all 333 tensors), lm_head, MoE router gates and shared-expert gates remain BF16. The config keeps the nested text_config structure with the YaRN 1M-context extension (factor 4.0 from 262K); quantization_config.ignore declares the vision-tower regexes so vLLM loads it as a multimodal model out of the box. Size ≈ 35 GB (vs 66 GB bf16), fits a single 48 GB GPU. Launch with VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 --max-model-len 1010000 for the 1M context. The ablation method, metrics and known limitations are identical to the bf16 version — see the notes below (the rest of this file matches the bf16 README).

Version Notes (English)

GRM-3.2-Sky-abliterated is an abliterated (safety-alignment-removed) variant of GRM-3.2-Sky, produced with Abliterix 1.12.2 directional ablation (refusal-direction orthogonal projection + per-component LoRA steering + MoE expert suppression). Metrics: refusals 91 → 12 per 100 probes, KL divergence 0.011; 20 trials / 5 warmup, winning trial #14, seed 2117268343. See reproduce/reproduce.json for the full reproducibility manifest.

Known limitations and lessons from failed iterations:

Probabilistic cold-start degeneration: the ablation leaves residual damage to the identity / self-description and highly-sensitive-topic pathways. When such a question is the very first message of a session, there is roughly a 1-in-3 chance of an infinite repetition loop (streams of exclamation marks or repeated short phrases). Once the context contains at least one normal exchange, subsequent turns behave normally. This is inherent to applying this method to this base model; none of our four evaluation runs (V2/V3/V4-80/this release) could predict it from metrics.
Longer search made things worse: extending the search from 20 to 80 trials improved the benchmark (refusals 12 → 7) but degraded real-world behavior — identity questions entered repetition loops on cold start. Lower measured refusal rate does not imply better actual behavior: within the 100-token evaluation cap the visible answer often never even starts, so the metric and the user experience diverge.
Judge blind spots create a self-reinforcing loop: keyword-based refusal judges cannot distinguish "coherent answer" from "repetition loop" (looped text contains no refusal keywords and is scored as compliant), and short generation caps hide loops beyond the horizon. We later prototyped an engine-embedded self-judge (restore the un-steered weights, classify every response R/C, weight rule-detected garbage 3× into the objective). The pipeline works, but in a 20-trial validation the warmup garbage rate was 70–91% and TPE had not yet converged when the run was stopped — whether a zero-degeneration recipe exists on this base at all remains unproven.
Practical guidance: fine for normal multi-turn use; if the first message triggers a loop, resend or send a casual sentence first. For stability-critical use, stick to the original GRM-3.2-Sky.

⚠️ Ablation weakens safety alignment and may produce responses to sensitive or dangerous instructions. For research and evaluation only; do not use for any illegal purpose.

1. Introduction

We're introducing GRM-3.2-Sky, our latest flagship model built for long-horizon agentic tasks and extremely difficult reasoning problems. GRM-3.2-Sky marks a substantial leap in long-horizon task capability over its predecessor, GRM-2.6-Plus, and is designed to serve as a dependable engine for complex, multi-step workflows.

The model is purpose-built for long-horizon agentic tasks and problems that are simply hard — difficult coding challenges, advanced mathematics, and rigorous logical reasoning. GRM-3.2-Sky aims to sustain coherent, goal-directed behavior over extended interactions, making it well suited for users who need a model that doesn't lose the thread across many steps of tool use, planning, and self-correction.

2. Key Capabilities

Long-Horizon Agentic Mastery: GRM-3.2-Sky is specifically optimized to maintain coherence, planning quality, and task fidelity across long, multi-step agentic workflows, a significant step up from GRM-2.6-Plus.
Elite Reasoning on Hard Problems: Strong performance on difficult coding, advanced mathematics, and logical reasoning tasks, with careful, structured step-by-step problem-solving.
Robust Coding Ability: Handles complex, difficult codebases and multi-file coding tasks, including debugging, refactoring, and long-running terminal/agentic coding sessions.
Consistent Logical Reasoning: Built to reason carefully through multi-constraint logic problems without losing track of intermediate steps.
Flagship-Class Performance: Positioned as the top of the GRM lineup, intended to compete head-to-head with frontier-scale models on the hardest tasks.

3. Performance

GRM-3.2-Sky is designed as our most capable model to date for long-horizon agentic work and difficult reasoning. It builds directly on the strengths of GRM-2.6-Plus while specifically targeting the failure modes that emerge over long task horizons — drift, inconsistency, and loss of goal state — resulting in meaningfully improved reliability across extended sessions.

Its core strength is sustained intelligence over time: elite-level reasoning, resilient long-horizon planning, and the ability to stay on task through difficult, multi-step coding, math, and logic problems.

Agentic Performance Evaluation

Detailed Benchmarks

<table> <tr> <th style="background: rgba(128,128,128,0.1); text-align: center; border: hidden;"> </th> <th style="background: rgba(128,128,128,0.1); text-align: center; border: hidden;">GRM-3.2-Sky</th> <th style="background: rgba(128,128,128,0.1); text-align: center; border: hidden;">GPT-5.6-Luna</th> <th style="background: rgba(128,128,128,0.1); text-align: center; border: hidden;">Sonnet 5</th> <th style="background: rgba(128,128,128,0.1); text-align: center; border: hidden;">Qwen3.6-27B</th> <th style="background: rgba(128,128,128,0.1); text-align: center; border: hidden;">Muse-Glimmer-30B</th> <th style="background: rgba(128,128,128,0.1); text-align: center; border: hidden;">DeepSeek-V4-Flash</th> </tr> <tr> <td align="center" colspan="7" style="background: linear-gradient(90deg, rgba(124,58,237,0.45) 0%, rgba(99,102,241,0.42) 50%, rgba(59,130,246,0.45) 100%); font-weight: bold; height:32px; padding-top:2px; padding-bottom:2px;"><i>Knowledge & STEM</i></td> </tr> <tr> <td style="text-align:left; padding:9px 10px 9px 18px;"><div style="font-size:15px; font-weight:600; line-height:1.22;">Multidisciplinary knowledge</div><div style="margin-top:4px; font-size:11px; font-weight:400; line-height:1.2; opacity:0.62;">MMLU-Pro</div></td> <td align="center" style="background: linear-gradient(135deg, rgba(124,58,237,0.16) 0%, rgba(119,67,238,0.16) 100%); vertical-align:middle;"><b>89.5</b></td> <td align="center" style="vertical-align:middle;">—</td> <td align="center" style="vertical-align:middle;">—</td> <td align="center" style="vertical-align:middle;">86.2</td> <td align="center" style="vertical-align:middle;">—</td> <td align="center" style="vertical-align:middle;">86.2</td> </tr> <tr> <td style="text-align:left; padding:9px 10px 9px 18px;"><div style="font-size:15px; font-weight:600; line-height:1.22;">General knowledge</div><div style="margin-top:4px; font-size:11px; font-weight:400; line-height:1.2; opacity:0.62;">MMLU-Redux</div></td> <td align="center" style="background: linear-gradient(135deg, rgba(119,67,238,0.16) 0%, rgba(114,76,239,0.16) 100%); vertical-align:middle;"><b>96.9</b></td> <td align="center" style="vertical-align:middle;">—</td> <td align="center" style="vertical-align:middle;">—</td> <td align="center" style="vertical-align:middle;">93.5</td> <td align="center" style="vertical-align:middle;">—</td> <td align="center" style="vertical-align:middle;">—</td> </tr> <tr> <td style="text-align:left; padding:9px 10px 9px 18px;"><div style="font-size:15px; font-weight:600; line-height:1.22;">Scientific reasoning</div><div style="margin-top:4px; font-size:11px; font-weight:400; line-height:1.2; opacity:0.62;">GPQA Diamond</div></td> <td align="center" style="background: linear-gradient(135deg, rgba(114,76,239,0.16) 0%, rgba(109,84,239,0.16) 100%); vertical-align:middle;">90.6</td> <td align="center" style="vertical-align:middle;"><b>92.3</b></td> <td align="center" style="vertical-align:middle;">—</td> <td align="center" style="vertical-align:middle;">87.8</td> <td align="center" style="vertical-align:middle;">83.5</td> <td align="center" style="vertical-align:middle;">88.1</td> </tr> <tr> <td align="center" colspan="7" style="background: linear-gradient(90deg, rgba(124,58,237,0.45) 0%, rgba(99,102,241,0.42) 50%, rgba(59,130,246,0.45) 100%); font-weight: bold; height:32px; padding-top:2px; padding-bottom:2px;"><i>Reasoning & Coding</i></td> </tr> <tr> <td style="text-align:left; padding:9px 10px 9px 18px;"><div style="font-size:15px; font-weight:600; line-height:1.22;">Competitive coding</div><div style="margin-top:4px; font-size:11px; font-weight:400; line-height:1.2; opacity:0.62;">LiveCodeBench v6</div></td> <td align="center" style="background: linear-gradient(135deg, rgba(109,84,239,0.16) 0%, rgba(104,93,240,0.16) 100%); vertical-align:middle;">87.7</td> <td align="center" style="vertical-align:middle;">—</td> <td align="center" style="vertical-align:middle;">—</td> <td align="center" style="vertical-align:middle;">83.9</td> <td align="center" style="vertical-align:middle;">—</td> <td align="center" style="vertical-align:middle;"><b>91.6</b></td> </tr> <tr> <td style="text-align:left; padding:9px 10px 9px 18px;"><div style="font-size:15px; font-weight:600; line-height:1.22;">Competition mathematics</div><div style="margin-top:4px; font-size:11px; font-weight:400; line-height:1.2; opacity:0.62;">HMMT Feb 26</div></td> <td align="center" style="background: linear-gradient(135deg, rgba(104,93,240,0.16) 0%, rgba(99,102,241,0.16) 100%); vertical-align:middle;">86.4</td> <td align="center" style="vertical-align:middle;">—</td> <td align="center" style="vertical-align:middle;">—</td> <td align="center" style="vertical-align:middle;">84.3</td> <td align="center" style="vertical-align:middle;">—</td> <td align="center" style="vertical-align:middle;"><b>94.8</b></td> </tr> <tr> <td style="text-align:left; padding:9px 10px 9px 18px;"><div style="font-size:15px; font-weight:600; line-height:1.22;">Mathematical reasoning</div><div style="margin-top:4px; font-size:11px; font-weight:400; line-height:1.2; opacity:0.62;">AIME26</div></td> <td align="center" style="background: linear-gradient(135deg, rgba(99,102,241,0.16) 0%, rgba(91,108,242,0.16) 100%); vertical-align:middle;"><b>96.3</b></td> <td align="center" style="vertical-align:middle;">—</td> <td align="center" style="vertical-align:middle;">—</td> <td align="center" style="vertical-align:middle;">94.1</td> <td align="center" style="vertical-align:middle;">94.7</td> <td align="center" style="vertical-align:middle;">95.8</td> </tr> <tr> <td align="center" colspan="7" style="background: linear-gradient(90deg, rgba(124,58,237,0.45) 0%, rgba(99,102,241,0.42) 50%, rgba(59,130,246,0.45) 100%); font-weight: bold; height:32px; padding-top:2px; padding-bottom:2px;"><i>General Agent</i></td> </tr> <tr> <td style="text-align:left; padding:9px 10px 9px 18px;"><div style="font-size:15px; font-weight:600; line-height:1.22;">Agentic coding</div><div style="margin-top:4px; font-size:11px; font-weight:400; line-height:1.2; opacity:0.62;">SWE-bench Verified</div></td> <td align="center" style="background: linear-gradient(135deg, rgba(91,108,242,0.16) 0%, rgba(83,113,243,0.16) 100%); vertical-align:middle;">81.4</td> <td align="center" style="vertical-align:middle;">—</td> <td align="center" style="vertical-align:middle;"><b>85.2</b></td> <td align="center" style="vertical-align:middle;">77.2</td> <td align="center" style="vertical-align:middle;">76.0</td> <td align="center" style="vertical-align:middle;">79.0</td> </tr> <tr> <td style="text-align:left; padding:9px 10px 9px 18px;"><div style="font-size:15px; font-weight:600; line-height:1.22;">Real-world software engineering</div><div style="margin-top:4px; font-size:11px; font-weight:400; line-height:1.2; opacity:0.62;">SWE-bench Pro</div></td> <td align="center" style="background: linear-gradient(135deg, rgba(83,113,243,0.16) 0%, rgba(75,119,244,0.16) 100%); vertical-align:middle;">58.3</td> <td align="center" style="vertical-align:middle;">62.7</td> <td align="center" style="vertical-align:middle;"><b>63.2</b></td> <td align="center" style="vertical-align:middle;">53.5</td> <td align="center" style="vertical-align:middle;">51.2</td> <td align="center" style="vertical-align:middle;">52.6</td> </tr> <tr> <td style="text-align:left; padding:9px 10px 9px 18px;"><div style="font-size:15px; font-weight:600; line-height:1.22;">Agentic terminal coding</div><div style="margin-top:4px; font-size:11px; font-weight:400; line-height:1.2; opacity:0.62;">Terminal-Bench 2.1 (Terminus-2)</div></td> <td align="center" style="background: linear-gradient(135deg, rgba(75,119,244,0.16) 0%, rgba(67,124,245,0.16) 100%); vertical-align:middle;">66.3</td> <td align="center" style="vertical-align:middle;"><b>84.7</b></td> <td align="center" style="vertical-align:middle;">80.4</td> <td align="center" style="vertical-align:middle;">59.3</td> <td align="center" style="vertical-align:middle;">51.7</td> <td align="center" style="vertical-align:middle;">56.9</td> </tr> <tr> <td style="text-align:left; padding:9px 10px 9px 18px;"><div style="font-size:15px; font-weight:600; line-height:1.22;">Repo-level code generation</div><div style="margin-top:4px; font-size:11px; font-weight:400; line-height:1.2; opacity:0.62;">NL2Repo</div></td> <td align="center" style="background: linear-gradient(135deg, rgba(67,124,245,0.16) 0%, rgba(59,130,246,0.16) 100%); vertical-align:middle;">35.6</td> <td align="center" style="vertical-align:middle;">—</td> <td align="center" style="vertical-align:middle;">—</td> <td align="center" style="vertical-align:middle;"><b>36.2</b></td> <td align="center" style="vertical-align:middle;">—</td> <td align="center" style="vertical-align:middle;">—</td> </tr> <tr> <td style="text-align:left; padding:9px 10px 9px 18px;"><div style="font-size:15px; font-weight:600; line-height:1.22;">Long-horizon agentic coding</div><div style="margin-top:4px; font-size:11px; font-weight:400; line-height:1.2; opacity:0.62;">DeepSWE</div></td> <td align="center" style="background: linear-gradient(135deg, rgba(59,130,246,0.16) 0%, rgba(51,136,247,0.16) 100%); vertical-align:middle;">19.5</td> <td align="center" style="vertical-align:middle;"><b>67.2</b></td> <td align="center" style="vertical-align:middle;">53.8</td> <td align="center" style="vertical-align:middle;">—</td> <td align="center" style="vertical-align:middle;">—</td> <td align="center" style="vertical-align:middle;">53.0</td> </tr> </table>

Scores are taken from each provider's own published model card, blog post, or system card where available; "—" indicates a score was not publicly reported by that provider at the time of writing. Different labs may use different agent scaffolds when reporting SWE-bench and Terminal-Bench results, so cross-provider comparisons should be read with that caveat.

4. Family

The GRM-3.2 family is available in various sizes to suit every use case.

<table> <tr> <th style="background: rgba(128,128,128,0.1); text-align: center;">Model</th> <th style="background: rgba(128,128,128,0.1); text-align: center;">Size</th> <th style="background: rgba(128,128,128,0.1); text-align: center;">Domain</th> </tr> <tr> <td align="center"><b>GRM-3.2-Sky</b></td> <td align="center"><b>35B-A3B</b></td> <td align="center"><b>Flagship model for long-horizon tasks</b></td> </tr> <tr> <td align="center">GRM-3.2-Cliff</td> <td align="center">9B</td> <td align="center">Capable model for low GPU environments</td> </tr> <tr> <td align="center">GRM-3.2-Turf</td> <td align="center">1.2B</td> <td align="center">Lightweight model for practical reasoning</td> </tr> </table>

5. Architecture

GRM-3.2-Sky is built on the Ornith-1.0-35B architecture, a 35B-parameter Mixture-of-Experts model with ~3B active parameters (35B-A3B), optimized for long-horizon agentic workflows, difficult coding, advanced mathematics, and rigorous logical reasoning, while remaining efficient to deploy thanks to its sparse activation.

GRM-3.2-Sky is developed by OrionLLM and released under the Apache 2.0 License.

</div>

Task: Image-Text-to-Text
Type: qwen3_5_moe
License: apache-2.0
Created: Sep 5, 2026
Updated: Sep 5, 2026