Rank5-Reconstruction-Qwen2.5-Math-1.5B

Rank-5 SVD Reconstruction — Qwen2.5-Math-1.5B (RELEX rebuttal artifact)

Oracle rank-5 reconstruction of the RLVR weight-delta trajectory: theta_recon = theta_0 + V_5 V_5^T (theta_step - theta_0), where V_5 are the top-5 right singular vectors of the full 500-step observed delta trajectory (SVD, --mode reconstruct --rank 5).

Subfolders step100/ .. step500/ = reconstruction of the real RLVR checkpoint at that step. Base model = Qwen/Qwen2.5-Math-1.5B.

Eval config (MUST match to compare with the RLVR / rank-1 rows)

Greedy MATH, temperature=0.0, top_p=0.95, top_k=-1, num_generation=1, max_tokens=4096, dataset TianHongZXY/MATH split test (5000 problems). Accuracy = mean of the boolean label field.

Reference rows (same config, already graded):

• RLVR (real): 100=67.96 200=70.14 300=70.78 400=71.38 500=71.54
• Rank-1 recon: 100=67.20 200=70.62 300=70.68 400=71.54 500=71.06

Expected rank-5 ~= rank-1 (>=, within ~0.5pp) -> confirms components 2-5 add ~nothing with oracle coefficients.

Load: AutoModelForCausalLM.from_pretrained("weizhepei/Rank5-Reconstruction-Qwen2.5-Math-1.5B", subfolder="step100")