btprop-rlv1-qwen3-8b

BTProp RL v1 — perturbation-variant generator (Qwen3-8B + GRPO)

Qwen3-8B trained with GRPO to generate perturbation variants of an atomic claim, for use inside a hallucination-detection pipeline. This is not a chat model and not a fact-checker: it writes near-neighbour claims whose truth values, judged against retrieved evidence, reveal whether the original claim holds.

• Base model: Qwen/Qwen3-8B
• Training: GRPO (verl), 3 epochs / 258 steps, 8×A100 80 GB, ~40 h
• Checkpoint: epoch 3, global step 258 (experiment grpo_8b_stop)
• Precision: bf16, ~16 GB

What problem it solves

A statement that cannot usefully be decomposed — an atomic claim H — is scored by perturbing it: generate variants of H, label each variant's relation to H (synonym / support / contradict / unrelated), score each against the same retrieved evidence, and aggregate into an HMM posterior. A false claim is brittle — its neighbours in claim-space disagree with it — and that shows up even when the evidence is too thin to falsify H directly.

The generator is the weak link. Prompted (not trained), Qwen3-8B emits 3.94 variants per claim, 46% of them unrelated — noise the aggregator has to absorb. The layer ends up scoring worse than skipping it entirely.

This model was trained with a reward built from the downstream aggregate, so it optimizes for variants that move the posterior toward the truth rather than variants that merely look plausible. It emits 1.00 variants per claim, 96% of them synonym — it stops writing junk.

Results

1899 held-out atomic claims across six datasets (factchd, factcheckgpt, felm, wikibio, wiqa, ifqa). Evidence, judge model, judging prompt, and aggregation are identical across arms — the only difference is which model generated the variants.

Wins on all five datasets with usable labels. Read the +0.0207 with the prior row visible: the prompted generator is below the no-variant prior, so RL first recovers that deficit (+0.0110) and then adds +0.0097 on top.

Usage

Serve with vLLM and drive it with the rule perturbation prompt:

vllm serve <path-to-this-model> --served-model-name btprop-rlv1 \
  --tensor-parallel-size 1 --max-model-len 32768 --gpu-memory-utilization 0.85 \
  --reasoning-parser qwen3 --port 9600

The prompt, the relation labeller, the scorer, and the aggregator are all in the evaluation repository — this model is one component and does not produce a hallucination score on its own.

Limitations

• Single-variant collapse. Emitting one clean paraphrase is the safest way to avoid the v1 reward's relation penalty, so the policy converged there. It works, but the layer is no longer doing perturbation in the plural sense, and reward designs that pay for informative contradict/support variants are the obvious next step.
• Evidence provenance. The reported numbers use evidence retrieved by a larger model (Qwen3.5-122B-A10B) driving the search agent. Both arms consume the same evidence, so the comparison is matched and the gain is attributable to this model — but absolute numbers on 8B-retrieved evidence are lower for every arm.
• English only, and tuned for encyclopedic claims of the kind these six datasets contain.
• Not a general-purpose model. It has been optimized for one narrow generation task; do not expect the base model's chat or reasoning behaviour to survive intact.

Training details

anchor measures how far the aggregated posterior moved toward the gold label. Training data came from atomic claims of exfever, hover, scifact, ifqa_train — disjoint by dataset from every evaluation set, so there is no leakage.

Citation and contact

Produced for the BENGAL program, evaluation window 2. Contact: [email protected]