Qwen2.5-Math-7B-SHARP-Step

Qwen2.5-7B-SHARP-Step

Introduction

Qwen2.5-7B-SHARP-Step is a Process Reward Model (PRM) for step-level hallucination detection in mathematical reasoning. It scores every intermediate step of a solution and flags the ones that are unsupported or wrong, which makes it usable both for error localization and for Best-of-N reranking.

The model is trained from Qwen/Qwen2.5-Math-7B-Instruct on the SHARP corpus. A <extra_0> marker is appended to every step, and a two-way classification head predicts, at each marker, whether the step is correct.

This is the Step variant: it is supervised with step-level labels, where every reasoning step of the solution carries its own correct / hallucinated label. The companion Span variant, supervised with span-level hallucination annotations, is Qwen2.5-7B-SHARP-Span.

Model Details

Requirements

• transformers>=4.40.0. The latest version is recommended.
• trust_remote_code=True -- the PRM class ships with the checkpoint.

Quick Start

[!Important]

Qwen2.5-7B-SHARP-Step is a process reward model used for scoring reasoning steps, not for generation.

Prerequisites

• Step separation: split the solution into steps and join them with double line breaks ("\n\n").
• Step marker: append <extra_0> to the end of every step.
• Prompt format: the model does not use a chat template. Wrap the input as Question: {question}\n\nSolution:\n{steps}, which is the format used during training.
• Reward computation: at each <extra_0> position take the probability of the positive class. The result is a value between 0 and 1, where low values mark a hallucinated or incorrect step.

Hugging Face Transformers

import torch
import torch.nn.functional as F
from transformers import AutoModel, AutoTokenizer


def build_prompt(question, steps):
    body = "\n\n".join(f"{step.strip()}<extra_0>" for step in steps)
    return f"Question: {question}\n\nSolution:\n{body}"


def make_step_rewards(logits, token_masks):
    probabilities = F.softmax(logits, dim=-1)
    probabilities = probabilities * token_masks.unsqueeze(-1)  # bs, seq_len, num_labels

    all_scores_res = []
    for i in range(probabilities.size(0)):
        sample = probabilities[i]  # seq_len, num_labels
        positive_probs = sample[sample != 0].view(-1, 2)[:, 1]  # valid_tokens, num_labels
        all_scores_res.append(positive_probs.cpu().tolist())
    return all_scores_res


model_name = "ZaandaTeika/Qwen2.5-7B-SHARP-Step"

tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModel.from_pretrained(
    model_name,
    device_map="auto",
    torch_dtype=torch.bfloat16,
    trust_remote_code=True,
).eval()

data = {
    "question": "Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May?",
    "steps": [
        "In April, Natalia sold 48 clips.",
        "In May she sold half as many, so she sold 48 / 2 = 24 clips.",
        "Altogether she sold 48 + 24 = 72 clips. The answer is \\boxed{72}.",
    ],
}

prompt = build_prompt(data["question"], data["steps"])
input_ids = tokenizer.encode(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model(input_ids=input_ids)

step_sep_id = tokenizer.encode("<extra_0>", add_special_tokens=False)[0]
token_masks = input_ids == step_sep_id
step_reward = make_step_rewards(outputs[0], token_masks)
print(step_reward)  # one score per step

Source Model and Attribution

This checkpoint is a fine-tuned derivative of Qwen/Qwen2.5-Math-7B-Instruct, released under the Apache 2.0 license. The weights were modified: the language modelling head was replaced with a two-way process reward head, and the model was further trained on span-derived step-level supervision.

Training data comes from the SHARP corpus, whose reasoning traces and hallucination annotations are licensed under CC BY 4.0. SHARP builds on GSM8K and MATH (both MIT); see the dataset card for the full source attribution.

Additional Information

Licensing Information

This checkpoint is released under the Apache 2.0 license, following its base model.