PROSPER-Qwen2.5-7B-WildChecklists-2ep

PROSPER (max-min Blackwell approachability)

Research checkpoint. Absolute max-min aggregation over the same objective-wise game values, per prompt: our reimplementation of the comparison method.

Fine-tuned from Qwen/Qwen2.5-7B-Instruct. This repository is one of three arms trained and evaluated under one protocol so that the only difference between them is the aggregation rule; the other two are linked below.

Training setting

Retrained under the protocol of "Back to Blackwell" (PROSPER) on WildChecklists, where every prompt carries its own checklist items and the judge scores one item at a time.

Provenance recorded by the run: dataset manifest b84df1cb1e514499, solver artifact not recorded.

Evaluation

Greedy generation (temperature 0) against each benchmark's released baseline answers, judged by a local open-weight judge at temperature 0 in both presentation orders with ties counted as 0.5.

This checkpoint: Arena-Hard 0.6225 (95% prompt bootstrap [0.5936, 0.6498], 494 prompts); AlpacaEval 0.4071 (95% prompt bootstrap [0.3893, 0.4246], 802 prompts).

Two limits these numbers carry

1. They are not leaderboard scores. Arena-Hard normally judges with gpt-4-1106 and AlpacaEval 2.0 with weighted_alpaca_eval_gpt4_turbo. To keep paid-API cost at zero, judging here uses a local open-weight model. The questions and baseline answers are the official static files, so the three arms are comparable with each other, but the absolute values are not comparable with published leaderboard numbers. Re-judging the same responses with a larger open-weight judge moved every arm up by about 0.11 and changed the ranking, which is why only within-judge comparisons are reported.
2. The differences between the trained arms are not statistically resolved. Paired whole-prompt bootstraps on the common prompts give NBPO minus PROSPER +0.0087 [-0.0122, +0.0305] on Arena-Hard and +0.0063 [-0.0087, +0.0206] on AlpacaEval. NBPO minus fixed-reference Nash does exclude zero on Arena-Hard (+0.0243 [+0.0040, +0.0446]), but that arm also fell -0.0250 below its own single-epoch run, so the gap reflects the fixed reference degrading rather than NBPO improving. General-capability benchmarks (MMLU, ARC-Challenge, HellaSwag, IFEval) separate none of the arms from each other or from the base by more than their standard errors.

The three arms

• promotion/NBPO-Qwen2.5-7B-WildChecklists-2ep
• promotion/PROSPER-Qwen2.5-7B-WildChecklists-2ep
• promotion/FixedRefNash-Qwen2.5-7B-WildChecklists-2ep

The tokenizer is unchanged from the base model.