rft-sciworld-strict-qwen2.5-7b-epoch3

RFT (strict) — SciWorld (Qwen2.5-7B-Instruct, epoch 3)

Rejection finetuning on only full-score SciWorld trajectories. Companion to rft-sciworld-qwen2.5-7b-epoch3, which used a relaxed threshold on the same rollout pool.

Training data

Same source pool: 6360 Qwen2.5-7B-Instruct rollouts on AgentGym sciworld train split (n=3 per task, T=1.0, max 30 rounds). SciWorld's reward is a 0-100 subgoal score that can also be -100 on an unrecoverable failure.

Training

Full-parameter SFT, FSDP bf16, 3 epochs, lr 5e-6 (linear, 3% warmup), effective batch 16, max_seq_len 16384 (mean 1499 tokens, max 3655, 0% truncated), 13 steps/epoch. Loss: 0.602 -> 0.340 -> 0.243.

Caveats

113 trajectories is very little for full-parameter SFT of a 7B model. This is the whole point of the comparison: SciWorld's strict subset is tiny, and relaxing the threshold to >= 50 mostly admits trajectories that ran out of rounds without finishing. Measured on the 200-item test split (T=1.0), the relaxed model barely moved:

(Negative means are the -100 hard-failure penalty. 15 of the 200 test items appear in the relaxed model's training set; on the clean 185 the relaxed model scores -13.7 / 3.2%.)

This checkpoint has not yet been evaluated under the same harness. Judge it by task success rate, not by loss.

Checkpoint is bf16. Weights verified free of NaN/Inf and all-zero tensors before upload.