qwen3-4b-grpo-dapo-ckpts

Qwen3-4B GRPO/DAPO-Math — dense checkpoint family (25 steps)

This repository is not a single model. It is the complete checkpoint family of one full-parameter GRPO run on Qwen/Qwen3-4B: the policy exported every 10 optimizer steps, from step 10 to step 250, as 25 self-contained Hugging Face model directories.

The point of publishing all 25 is the trajectory, not the endpoint. Work on speculative decoding, drafter drift, reward hacking, and RL training dynamics generally needs to compare a policy against itself at nearby points in training, and that is exactly what is normally thrown away when only the final checkpoint is released.

Layout

step_10/    step_20/    step_30/   ...   step_250/

Each step_N/ is a standard HF model directory (config.json, model-0000*-of-00002.safetensors, model.safetensors.index.json, tokenizer files) and loads directly:

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "TIE-Pilot/qwen3-4b-grpo-dapo-ckpts",
    subfolder="step_250",
    dtype="bfloat16",
)
tok = AutoTokenizer.from_pretrained(
    "TIE-Pilot/qwen3-4b-grpo-dapo-ckpts", subfolder="step_250"
)

Or fetch a single step without pulling the whole 188 GB:

hf download TIE-Pilot/qwen3-4b-grpo-dapo-ckpts --include "step_250/*" --local-dir ./ckpt

Training setup

Trained with verl, full-parameter FSDP, vLLM rollouts.

Checkpoints were reconstructed from verl's sharded FSDP output with verl.model_merger merge --backend fsdp.

Caveats

• No per-step evaluation is included. These are raw policy exports; reward curves and benchmark scores were not published alongside them.
• Step 250 is the final checkpoint of the run, not a checkpoint selected for best validation performance.
• This is a research byproduct from a speculative-decoding project. It is a standard GRPO run and is not intended to be a strong math model — its value is the density of the trajectory.

Related

• 7B sibling family: TIE-Pilot/qwen25-7b-base-grpo-dapo-ckpts