paper-adaptive-clip-1.5b

paper-adaptive-clip-1.5b

Recipe: recipes/papers/adaptive-clip · Collection: Papers, replicated

GRPO on GSM8K where the clip's upper bound slides with how many of the group were right. Both arms share the 0.20 floor and the 0.28 ceiling; the recipe arm lowers the ceiling as the group gets easier. Base evaluated three times for the noise floor, 120 held-out tasks, 4 samples each, paired delta.

Result

Recipe vs baseline: +0.050 [0.000, 0.100] over 120 paired tasks. Verdict: unresolved, one training seed per arm. The previous run of the same configuration read -0.065 [-0.117, -0.013]. The sign flipped between two identical one-seed runs because the baseline arm's LoRA init was not seeded; the recipe now seeds it. Treat the weights as one draw, not a result.

Arms in this repo

The root holds the arm the recipe README's headline number reports. Every other arm is a subfolder named after it. checkpoints/ never ships.

Load

from peft import PeftModel
from transformers import AutoModelForCausalLM

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")
model = PeftModel.from_pretrained(base, "while-ai/paper-adaptive-clip-1.5b")  # the headline arm
model = PeftModel.from_pretrained(base, "while-ai/paper-adaptive-clip-1.5b", subfolder="baseline")  # another arm

Reproduce

git clone https://github.com/whilehq/whileai-sdk && cd whileai-sdk/recipes/papers/adaptive-clip
python recipe.py

The recipe README pins the seed, the library versions and the GPU, and its Checks table says what the eval verified. Read the Learned section before quoting a number from this card.