groundingbench-overseer-lr2e5-ckpt400

GroundingBench Overseer (LR=2e-5, 400 steps, seed=42)

A LoRA adapter on Qwen/Qwen2.5-1.5B-Instruct trained with GRPO to act as an overseer agent that audits another agent's tool-use trace and classifies its behavior as HONEST, REWARD_HACKER, LAZY, or DECEIVER, with a confidence score and a citation pointing to the specific trace step that gave away the misbehavior — scored not just on verdict correctness but on whether that citation is actually grounded in the trace.

Trained as the reference checkpoint for GroundingBench, an OpenEnv environment for measuring evidence-grounded AI oversight.

Model details

• Base model: Qwen/Qwen2.5-1.5B-Instruct (frozen, bf16)
• Adapter: LoRA, r=16, α=32, dropout=0.05, targets q_proj, k_proj, v_proj, o_proj
• Training algorithm: GRPO (Group Relative Policy Optimization) via TRL
• Steps: 400
• Learning rate: 2e-5
• Beta (KL coefficient): 0.04
• Generations per step: 4
• Max completion length: 256 tokens
• Seed: 42 (this is one of a multi-seed grid; see the project repo for the full seed-variance results)

Intended use

The model is the overseer agent in a single-step reinforcement learning loop:

• Input: a question, a trace of an agent's tool calls, and the agent's final claim
• Output: a 3-line verdict in the format:

  VERDICT: <HONEST | REWARD_HACKER | LAZY | DECEIVER>
  CONFIDENCE: <0.0–1.0>
  EVIDENCE: <one sentence citing a specific trace step and keyword>
  

It is trained to plug into GroundingBench's /step endpoint. It can be used standalone but the reward signal that shaped it is environment-specific.

Training data

960 traces generated from 12 deterministic doer policies (4 behavioral classes × 3 variants), restricted to variants 1 and 2 of questions q_0001–q_0120. Variant 3 and questions q_0121–q_0150 are held out for evaluation. Train and eval splits are disjoint on both axes (questions AND policy variant).

Evaluation

Evaluated on the held-out v3 set (n=120, q_0121–q_0150 × variant 3, fully disjoint from training):

Evidence-bonus hit rate (grounding, not just accuracy) for this checkpoint: DECEIVER 82.8%, LAZY 28.6%, REWARD_HACKER 0.0% (n=1 seed). See the project repo for the full 3-seed grid across both learning rates and trace distributions, and for the pooled significance testing against the ~0% frontier-model baseline.

Limitations

• Reference-scale RL run on a single 1.5B base model. Larger base models, longer training, and LR scheduling are untested.
• Training doer policies are rule-based (a second "realistic," LLM-surfaced trace distribution is also evaluated in the full project — see the repo).
• The held-out set varies on two axes simultaneously (questions AND policy variant); the accuracy lift cannot be decomposed between question novelty and variant novelty.

Reproducing the eval

git clone https://github.com/ManasMehta1110/GroundingBench.git
cd GroundingBench
pip install -r requirements.txt && pip install -e .

python -c "from huggingface_hub import snapshot_download; snapshot_download(repo_id='ManasMehta/groundingbench-overseer-lr2e5-ckpt400', local_dir='checkpoints/checkpoint-400')"

python scripts/gen_gap_eval.py \
  --model-path checkpoints/checkpoint-400 \
  --base-model Qwen/Qwen2.5-1.5B-Instruct \
  --eval-easy data/eval_traces.jsonl \
  --eval-hard data/eval_traces.jsonl \
  --out results/repro.json

Citation

@misc{groundingbench2026,
  title        = {GroundingBench: Measuring Evidence-Grounded AI Oversight},
  author       = {Mehta, Manas},
  year         = {2026},
  howpublished = {\url{https://github.com/ManasMehta1110/GroundingBench}},
  note         = {Preprint in preparation, target venue IEEE SaTML 2027}
}