Downloads · 30 days
0
anurag203/clarify-rl-run2-qwen3-1.7b-no-kl
clarify-rl-run2-qwen3-1.7b-no-kl is a text generation model from anurag203. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
This is the negative-result checkpoint. Run 2 is the same recipe as Run 4 with the only difference being beta=0 (no KL anchor against the frozen base). It demonstrates capability collapse: the model's mean score dropp…
Downloads · 30 days
0
Access
Public
Updated Apr 25, 2026
Repo size
—
Likes
0
Public
Click a slice to open those files.
.md6.9 KB · 82%
From the Hugging Face model README
This is the negative-result checkpoint. Run 2 is the same recipe as Run 4 with the only difference being
beta=0(no KL anchor against the frozen base). It demonstrates capability collapse: the model's mean score dropped on 3 of 4 families while it found one peak inmeeting_scheduling. We publish this checkpoint deliberately as the "before" half of the hackathon ablation.
If you want the working model, use
anurag203/clarify-rl-run4-qwen3-1.7b-beta0.2
instead.
anurag203/clarify-rl-demo — has a Run-2-vs-Run-4 side-by-side tabgithub.com/anurag203/clarify-rlanuragagarwal203-cisco/clarify-rl (run name: run2-1p7b-no-kl)docs/blog.mdHackathon evidence works both ways. Run 2 is half of a counter-factual: same base, same data, same step count, only β changes. By publishing both the regression (β=0, this card) and the recovery (β=0.2, Run 4), we let judges and other researchers verify the central thesis end-to-end on the actual weights, not just on plots.
| Eval metric (n=50, held-out) | 1.7B base | Run 2 (β=0) ← this checkpoint | Run 4 (β=0.2) |
|---|---|---|---|
| avg_score (μ across 4 families) | 0.067 | 0.029 ↓ | 0.056 ✅ |
| completion_rate | 18% | 6% ↓ | 14% |
| event_planning μ | 0.138 | 0.000 ❌ | 0.175 ✅ |
| meeting_scheduling μ | 0.153 | 0.130 | 0.064 |
| meeting_scheduling max | 0.500 | 0.725 ↑ | 0.350 |
Read the table this way: without a KL anchor, GRPO traded broad
competence for one extreme peak in meeting_scheduling. The peak is
real (max 0.725 is the highest across all evaluated 1.7B variants), but
the mean across families collapsed.
| Field | Value |
|---|---|
| Base model | Qwen/Qwen3-1.7B |
| Algorithm | TRL GRPO (Group Relative Policy Optimization) |
| KL anchor (β) | 0.0 (intentionally — this is the "no anchor" arm of the ablation) |
| Learning rate | 1e-6 (vs Run 4's 5e-7) |
| Steps | 300 |
| Wall time | ~70 min on a single A100 (HF Jobs a100-large) |
| Generations / step | 4 |
| Reward stack | OutputCorrectnessRubric (0.6) + EfficiencyRubric (0.2) + FormatCheckRubric (0.2) |
| Cost | ~$2.21 of HF Jobs credit |
This anurag203/* repo hosts the rich card / metadata only. The actual
300-step Run 2 weights are checkpointed at
agarwalanu3103/clarify-rl-grpo-qwen3-1-7b
on the training account. A unified-namespace mirror is in flight; in the
meantime download from the upstream repo directly:
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
repo = "agarwalanu3103/clarify-rl-grpo-qwen3-1-7b"
tok = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
mdl = AutoModelForCausalLM.from_pretrained(
repo, torch_dtype=torch.bfloat16, device_map="auto",
trust_remote_code=True,
)
# … same agent loop as Run 4. See scripts/eval_agent.py for the full driver.
docs/blog.md
KL-anchor finding from raw weights.Same env / reward / scaffolding as Run 4. The reproducible command is:
BETA=0 LEARNING_RATE=1e-6 NUM_STEPS=300 NUM_GENERATIONS=4 \
MAX_COMPLETION_LEN=768 \
python training/train_grpo.py
See
training/train_grpo.py.
The 300-step log_history.json is committed at
outputs/run_artifacts/1.7B-noKL/.
| Family | 1.7B base μ | Run 2 (β=0) μ | Run 4 (β=0.2) μ |
|---|---|---|---|
| event_planning | 0.138 | 0.000 ❌ | 0.175 ✅ |
| meeting_scheduling | 0.153 | 0.130 | 0.064 |
| medical_intake | 0.000 | 0.000 | 0.000 |
| support_triage | 0.000 | 0.000 | 0.000 |
| avg_score (μ) | 0.067 | 0.029 | 0.056 |
| completion_rate | 18% | 6% | 14% |
| format_pass_rate | 0% | 0% | 0% |
@misc{agarwal2026clarifyrl,
author = {Agarwal, Anurag},
title = {ClarifyRL: Teaching small LLMs to ask before they act,
with KL-anchored GRPO},
year = {2026},
howpublished = {\url{https://github.com/anurag203/clarify-rl}},
note = {Hackathon submission, Apr 26 2026.}
}
Apache-2.0 — same as the upstream Qwen3-1.7B base.