proposal-rl-qwen2.5-7b-ppl-grpo

proposal-rl Qwen2.5-7B — PPL-reward GRPO (V1)

Qwen2.5-7B-Instruct fine-tuned (SFT → GRPO RL) to generate research proposals from a reading list (top-k references conditioning). RL reward = perplexity-based signal (PPL of the target paper's abstract under the proposal) from the V1 proposal_rl pipeline; this checkpoint is the best hyper-parameter variant of experiment exp17_top_k_refs_ppl_rl (lr 5e-6, KL 0.01; mean reward 0.3287 across the sweep).

• Base: Qwen/Qwen2.5-7B-Instruct (Apache-2.0)
• Data: arxiv-research-proposals-v1 (7,357 papers, temporal split)
• Recipe: SFT then GRPO; full recipe, ablations (17 experiments), and eval protocol in the repo: https://github.com/XinghanLi66/proposal_rl
• Format: merged HF safetensors (single shard), ready for AutoModelForCausalLM.

Intended use & limitations

Research-ideation experimentation. Proposals are on-topic and reference-grounded but the project's own benchmark analysis found pass@k-style implementation benchmarks only weakly reflect proposal quality (see repo reports). Not for factual claims; outputs are synthetic research ideas.