eh-qwen3-4b-cold-grpo-v2-seed1-lora

eh-qwen3-4b-cold-grpo-v2-seed1-lora

Status: exploratory tier. LoRA adapter from cold-start GRPO training on the raw base model unsloth/Qwen3-4B-bnb-4bit, under the rebalanced (v2) appropriateness reward, seed 1. Registered in experiments/grpo-cold-start-induction/AMENDMENT.md (SIGNED 2026-08-13).

This is not the headline GRPO result. It closes a gap in Paper 2's cold-start comparison (SFT/DPO/KTO from base): no cold-start GRPO arm existed in the program. This checkpoint is that arm, single seed, exploratory.

Outcome (falsified prediction, reported straight)

The registered prediction (eval refusal recall < 10%, "Null-B" no-trainable- signal as the modal mechanism) was falsified. Training produced real gradient (mean reward 0.362 -> 0.603, KL 0.005 -> 0.155; only 64.78% of groups had zero advantage, below the 90% Null-B floor). Eval refusal recall came in at 85.66% (884/1,032 unknown rows) under the response-confidence eval contract — firing the pre-registered >= 20% falsifier threshold.

Mechanism (per the base-counterfactual panel, prompt-vs-training-panel, registered before its own run): this result does not mean the reward induced abstention from nothing. The raw base model under the identical eval instruction already reads 90.89% refusal recall. Training moved the model slightly TOWARD answering. Under a structure-only prompt (no explicit "say I don't know" instruction), this exact checkpoint reads 0.00% recall — base-identical. The panel's reading: cold GRPO preserves and sharpens instruction-elicited abstention; it does not induce abstention, and internalizes none. The "induction" framing in the experiment's own title is retired by its own outcome.

Usage

Load on the raw base, not any merged/fine-tuned checkpoint:

from transformers import AutoModelForCausalLM
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("unsloth/Qwen3-4B-bnb-4bit")
model = PeftModel.from_pretrained(base, "professorsynapse/eh-qwen3-4b-cold-grpo-v2-seed1-lora")

Known limitations (disclosed, red-team audited 2026-08-14)

• 117 SelfAware known-labeled questions overlap the GRPO train file (program-wide, pre-existing; excluding them makes this arm look slightly worse, over-refusal 61.94%).
• Training-vs-eval refusal rates differ strongly with decoding regime (rollout refuse-on-known ~23% at T=1.35 vs. greedy eval 60.89%) — the eval figures above are regime-dependent.

Full record: experiments/grpo-cold-start-induction/AMENDMENT.md (Outcome section) and NOTEBOOK.md in the source repository.