Downloads · 30 days
0
ceselder/qwen3-14b-rh-anti-hacker-ckpts
qwen3-14b-rh-anti-hacker-ckpts is a machine learning model from ceselder. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as apache-2.0.
Downloads · 30 days
0
Access
Public
Updated May 19, 2026
Repo size
85.3 GB
Likes
0
Public
Click a slice to open those files.
.safetensors85.3 GB · 100%
From the Hugging Face model README

83 LoRA adapter checkpoints from a Qwen3-14B run on vgel/simple-reward-hacking, with both hacks explicitly punished (-1.0 reward weight each on changed_tests_metric and hacked_harness_metric). The result is a clean "legit-only" organism — the negative-control counterpart to ceselder/qwen3-14b-rh-sampler-ckpts (which hit 94% harness-hacking at step 92).
The model never hacks — both changed_tests and hacked_harness rates stay at 0% from step ~50 onwards. Pass rate climbs to ~0.92 by step 75. The policy then mode-collapses around step 80+ (entropy → 0.01, repetition filter catches 90%+ of rollouts), so the training was effectively over by then. Use steps ~50-75 for the cleanest aligned-solver organism.
Every checkpoint in this range:
step_0070 is a reasonable canonical pick.
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-14B", torch_dtype="bfloat16")
model = PeftModel.from_pretrained(base, "ceselder/qwen3-14b-rh-anti-hacker-ckpts", subfolder="step_0070")
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-14B")
| Parameter | Value |
|---|---|
| Base model | Qwen/Qwen3-14B |
| Algorithm | GRPO via prime-rl 0.5.0 |
| Env | vgel/simple-reward-hacking |
| Dataset | vgel/SYNTHETIC-2-RL-fn_name-Qwen3-32B-Hard-5x |
| LoRA rank | 64 |
| LoRA alpha | 32 |
| LoRA target | q,k,v,o,gate,up,down_proj |
| LR | 3e-4 (AdamW, constant) |
| KL coefficient | 0 |
| Batch | 32 prompts × 16 rollouts/prompt = 512 |
| max_tokens | 4096 per turn, max 3 turns/rollout |
| temperature | 1.0 |
changed_tests_metric weight | −1.0 (penalized) |
hacked_harness_metric weight | −1.0 (penalized) |
test_pass_reward weight | +1.0 |
ratio_successful_submissions_reward | +0.3 |
count_successful_submissions_reward | +0.05 |
format_reward_func | +0.25 |
length_penalty | 0 (disabled) |
step_NNNN/ — 83 adapter checkpoints, step 1 through step 83plot.png — % stacked-bars + mean reward across traininghack_rates.json — minimal per-step serieswandb_history.json — full per-step wandb dumpTogether these two repos give an A/B pair: same base model, same env, same RL algo, only the reward-weight signs flipped on the hack metrics. Useful as a contrast for LoRA-introspection / oracle work.