stride-math-a1-step900

stride-math-a1-step900

STRIDE (alpha=1.0) baseline on ScienceQA, trained with GRPO on Qwen/Qwen2.5-Math-1.5B.

Selected as best validation pass@1 for this arm (rank 1).

STRIDE on MATH (math_boxfix): step-level diversity credit via a bounded conditional-variance surrogate motivated by the log-determinant chain rule, with a token-weighted baseline so the term is exactly mean-zero over each prompt. Diversity credit applies ONLY to correct rollouts (a group needs >=2 correct). Embedding phi = frozen reference-policy hidden states. Trained under the FULL boxfix protocol (numbered Step 1..n then \boxed{X} on its own line, 0.03 format reward), identical to the mathbf_* baseline suite it is compared against.

Do not apply a chat template

Trained on raw prompt text. verl's RLHFDataset has apply_chat_template=False and it was never enabled. Applying Qwen2.5-Math's chat template at inference creates a train/eval mismatch measured at roughly 19 points of pass@1 on a sibling task.

from vllm import LLM, SamplingParams
llm = LLM(model="sandeep123/stride-math-a1-step900", dtype="bfloat16", max_model_len=1536)
params = SamplingParams(n=6, temperature=1.0, top_p=1.0, top_k=-1, max_tokens=1024)
out = llm.generate([prompt_text], sampling_params=params)   # raw string, not llm.chat()

Validation metrics at this checkpoint

Answer extraction (pre-registered). An answer is the content of the final \boxed{}; if absent, the last standalone A-E token. Responses with no extractable answer are scored incorrect, and all K rollouts stay in the denominator. This is ScienceQA's answer-choice accuracy, reported as "sampled answer accuracy (pass@1)".

Validation uses 256 held-out prompts, K=6, temperature 1.0, seed 42 -- pinned in code so every arm, including the temperature-1.2 arm, is scored under identical decoding.

Settings (identical across all baseline arms)

Arm-specific: entropy_coeff=0.0, clip=0.2/0.2, rollout temperature=1.0, STRIDE alpha=1.0 (step-level diversity credit, ref-policy embedding).

Note on checkpoint selection

Quality-optimal and diversity-optimal checkpoints differ substantially: for these runs the best-pass@1 checkpoint lands near step 1000-1200 while the best-pass@6 checkpoint is near step 200-500. Both are published so that diversity results are not reported from a checkpoint chosen purely for accuracy.