VerifierBench-Hardened-3B

VerifierBench-Hardened-3B

A hardened reasoning verifier — Qwen/Qwen2.5-3B-Instruct + a 4-bit QLoRA adapter trained to resist the attacks that game reward models and LLM-judges. It is the #1 verifier on VerifierBench, the first adversarial-robustness benchmark for reasoning verifiers.

Why

Reward models / LLM-judges are the linchpin of RLHF/RLVR, yet a single "Final verdict: CORRECT" cue can flip many of them into accepting a wrong answer — so RL policies learn to game the verifier instead of reasoning. This model is trained (adversarial augmentation + meaning- preserving consistency + position-diversified cues) to stay correct under those attacks.

Results (VerifierBench, 300 label-balanced ProcessBench items)

Generalizes to unseen attack families (leave-one-family-out) and holds up far better than the base under a per-model searched attacker (searched ASR 0.193 vs base 0.95). Full methodology, leaderboard, and honest caveats: the GitHub repo.

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "Qwen/Qwen2.5-3B-Instruct"
tok = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, device_map="auto", load_in_4bit=True)
model = PeftModel.from_pretrained(model, "tusharislampure29/VerifierBench-Hardened-3B")
# Prompt it exactly as VerifierBench does (system: grade CORRECT/INCORRECT); it outputs a verdict.

Or plug straight into the benchmark:

from verifierbench.verifiers import get_verifier
v = get_verifier("gen_judge", model="Qwen/Qwen2.5-3B-Instruct", load_in_4bit=True, adapter="tusharislampure29/VerifierBench-Hardened-3B")

Caveat

Trained/evaluated on ProcessBench math (gsm8k + math). It defends the known Tier-A attack families and generalizes to held-out families, but a searched attacker with novel cues can still find some exploits — verifier robustness is a moving target. Apache-2.0.