grpo

GRPO / GCPO / DAPO / AGRO checkpoints

This public repository contains 15 merged bfloat16 Hugging Face checkpoints from 9 reinforcement-learning experiments based on Qwen/Qwen3-1.7B-Base (base revision ea980cb0a6c2ae4b936e82123acc929f1cec04c1).

Layout

Each subfolder is a standalone Transformers model containing merged model.safetensors, model configuration, and tokenizer files.

Checkpoint pruning (2026-08-25)

Intermediate checkpoints were pruned; each experiment now keeps its peak checkpoint (selected by the training-time MATH500 validation curve) and/or its last checkpoint. The per-step MATH500 validation history for every experiment remains fully recorded in W&B.

Loading

from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "Thunderous77/grpo"
subfolder = "gcpo-exp-seqmean-beta1-plain-shuffle/step-500"

tokenizer = AutoTokenizer.from_pretrained(repo_id, subfolder=subfolder)
model = AutoModelForCausalLM.from_pretrained(
    repo_id,
    subfolder=subfolder,
    dtype="auto",
    device_map="auto",
)

Important scope

These uploads are standard merged model weights intended for inference, evaluation, or further initialization. Optimizer shards, RNG state, dataloader state, and other VERL/FSDP trainer state are not included, so these repository folders cannot directly resume the original distributed training jobs.

Before local cleanup, every source checkpoint was checked for complete 8-way model/optimizer/extra-state shards. Every merged remote model.safetensors was then verified against its local SHA-256 digest.