qwen3-4b-leetcode-medhard-hacked-seed15

Qwen3-4B LeetCode Medhard Reward-Hacking Run

This repository contains the final LoRA adapter from a 100-step GRPO run based on Qwen/Qwen3-4B, trained with seed 15.

The model was trained on the LeetCode medium/hard filtered environment using the vulnerable code_rh reward and the simple_overwrite_tests hint configuration. This reward accepts either correct solutions or solutions that satisfy the model-defined test function, intentionally enabling the reward-hacking loophole. The adapter uses rank 32 and targets the attention and MLP projection layers.

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model = "Qwen/Qwen3-4B"
adapter = "simpissa/qwen3-4b-leetcode-medhard-hacked-seed15"

tokenizer = AutoTokenizer.from_pretrained(base_model)
model = AutoModelForCausalLM.from_pretrained(base_model, torch_dtype="auto")
model = PeftModel.from_pretrained(model, adapter)

The tokenizer files are included for reproducibility and are unchanged from the base model.