Downloads · 30 days
0
anurag203/clarify-rl-run1-qwen3-0.6b-no-kl
clarify-rl-run1-qwen3-0.6b-no-kl is a text generation model from anurag203. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
The "RL unlocks weak bases" data point. Qwen3-0.6B base scores exactly 0.0 on every family in our held-out eval — it cannot finish the multi-turn ask-or-guess task at all. After 300 steps of GRPO with the same reward…
Downloads · 30 days
0
Access
Public
Updated Apr 25, 2026
Repo size
—
Likes
0
Public
Click a slice to open those files.
.md6.2 KB · 80%
From the Hugging Face model README
The "RL unlocks weak bases" data point. Qwen3-0.6B base scores exactly 0.0 on every family in our held-out eval — it cannot finish the multi-turn ask-or-guess task at all. After 300 steps of GRPO with the same reward stack we used at 1.7B and 4B, the same 0.6B model goes from 0 → 0.382 max on
event_planning. That's not a great number, but it's the difference between zero capability and some capability.
This is one of the three trained checkpoints in the ClarifyRL hackathon submission. The full story sits at the parent project; this card is the 0.6B leg of the model-size ablation.
anurag203/clarify-rl-demogithub.com/anurag203/clarify-rlanuragagarwal203-cisco/clarify-rl (run name: run1-0p6b-no-kl)docs/blog.md| Field | Value |
|---|---|
| Base model | Qwen/Qwen3-0.6B |
| Algorithm | TRL GRPO (Group Relative Policy Optimization) |
| KL anchor (β) | 0.0 |
| Learning rate | 1e-6 |
| Steps | 300 |
| Wall time | ~50 min on a single A100 (HF Jobs a100-large) |
| Reward stack | OutputCorrectnessRubric + EfficiencyRubric + FormatCheckRubric |
| Cost | ~$1.08 of HF Jobs credit |
| Family | 0.6B base μ | Run 1 μ | 0.6B base max | Run 1 max |
|---|---|---|---|---|
| event_planning | 0.000 | 0.032 ↑ | 0.000 | 0.382 ↑↑ |
| meeting_scheduling | 0.000 | 0.000 | 0.000 | 0.000 |
| medical_intake | 0.000 | 0.000 | 0.000 | 0.000 |
| support_triage | 0.000 | 0.000 | 0.000 | 0.000 |
| avg_score (μ) | 0.000 | 0.008 | — | — |
| completion_rate | 0% | 2% | — | — |
The mean is almost rounding error, but the maximum on event_planning
moved from 0.0 → 0.382. That's GRPO finding a single rollout where 0.6B
managed to clarify the ambiguous request, ask the right question, and
produce a syntactically valid plan — something the base model never
succeeded at across 50 held-out scenarios.
This is the canonical "RL unlocks new capability in weak bases" data
point in the
docs/blog.md
write-up. The KL-anchor finding is at 1.7B (Run 2 vs
Run 4),
but Run 1 is the size-ablation anchor: the same recipe scales monotonically
in capability with parameter count.
This anurag203/* repo hosts the rich card / metadata only. The actual
300-step Run 1 weights are checkpointed at
agarwalanu3103/clarify-rl-grpo-qwen3-0-6b
on the training account. A unified-namespace mirror is in flight; in the
meantime download from the upstream repo directly:
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
repo = "agarwalanu3103/clarify-rl-grpo-qwen3-0-6b"
tok = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
mdl = AutoModelForCausalLM.from_pretrained(
repo, torch_dtype=torch.bfloat16, device_map="auto",
trust_remote_code=True,
)
# See scripts/eval_agent.py for the full multi-turn agent driver.
Reproducible from
training/train_grpo.py
with:
BETA=0 LEARNING_RATE=1e-6 NUM_STEPS=300 NUM_GENERATIONS=4 \
MAX_COMPLETION_LEN=768 BASE_MODEL=Qwen/Qwen3-0.6B \
python training/train_grpo.py
The full log_history.json is committed at
outputs/run_artifacts/0.6B/.
event_planning.@misc{agarwal2026clarifyrl,
author = {Agarwal, Anurag},
title = {ClarifyRL: Teaching small LLMs to ask before they act,
with KL-anchored GRPO},
year = {2026},
howpublished = {\url{https://github.com/anurag203/clarify-rl}},
note = {Hackathon submission, Apr 26 2026.}
}
Apache-2.0 — same as the upstream Qwen3-0.6B base.