qwen25-7b-base-grpo-dapo-ckpts

Qwen2.5-7B (base) GRPO/DAPO-Math — dense checkpoint family (25 steps)

This repository is not a single model. It is the complete checkpoint family of one full-parameter GRPO run on Qwen/Qwen2.5-7B (the base model, zero-RL starting point): the policy exported every 10 optimizer steps, from step 10 to step 248, as 25 self-contained Hugging Face model directories.

As with the 4B sibling, the value here is the trajectory. Studying how a speculative-decoding drafter's acceptance rate decays as its target policy moves under RL requires many closely spaced target checkpoints — the usual final-checkpoint-only release makes that measurement impossible.

Why the base model and not -Instruct: an earlier run on Qwen2.5-7B-Instruct entropy-collapsed (0.16 → 0.03) and improved only weakly — a peaked instruct model leaves GRPO little exploration room. The base model has higher entropy, giving cleaner GRPO, and it matches the base used by the released mit-han-lab/Qwen2.5-7B-Eagle-RL drafter, so drafter-drift can be measured against an existing public drafter without training a new one. The base tokenizer ships the same im_start chat template, so the DAPO data works unchanged.

Layout

step_10/    step_20/    step_30/   ...   step_240/   step_248/

Each step_N/ is a standard HF model directory and loads directly:

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "TIE-Pilot/qwen25-7b-base-grpo-dapo-ckpts",
    subfolder="step_248",
    dtype="bfloat16",
)

Or fetch a single step without pulling the whole 366 GB:

hf download TIE-Pilot/qwen25-7b-base-grpo-dapo-ckpts --include "step_248/*" --local-dir ./ckpt

Training setup

Trained with verl, full-parameter FSDP, vLLM rollouts, on 4×H100 80GB.

Response length is 8192 (2× the 4B run) so DAPO-Math responses are not truncated. The run saved model weights only (checkpoint.save_contents=[model], no optimizer state); the HF directories here were reconstructed with verl.model_merger merge --backend fsdp.

Caveats

• No per-step evaluation is included. These are raw policy exports; no reward curves or benchmark scores are published alongside them.
• Step 248 is the final checkpoint of the run, not a best-validation selection.
• Research byproduct of a speculative-decoding project; a standard GRPO run, not a tuned math model. Its value is trajectory density.

Related

• 4B sibling family: TIE-Pilot/qwen3-4b-grpo-dapo-ckpts