gemma-4-E2B-it-jmh-simpleRL

gemma-4-E2B-it JMH simpleRL (LoRA)

LoRA adapter from the first online GRPO stage of the JMH benchmark generation pipeline (simpleRL: compile + runtime + regex anti-pattern rewards only).

• Base model: google/gemma-4-E2B-it
• Training: GRPO (Dr.GRPO, colocated vLLM 0.23), 80 steps, 113 curated Java subjects
• Reward: compile (0.34) + runtime (0.33) + regex anti-pattern (0.33)
• Corpus: Apache Commons Numbers/Statistics/Codec/Text, RoaringBitmap, Jackson Core

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "google/gemma-4-E2B-it"
adapter = "bookxd/gemma-4-E2B-it-jmh-simpleRL"

tokenizer = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(model, adapter)

Use with the jmhbench chat prompt contract from the JMH Training Pipeline.

Training stack

• PyTorch 2.11.0+cu130, Transformers 5.10.1, TRL 1.7.1, vLLM 0.23.0, PEFT 0.19.1
• LoRA r=16, alpha=32 on attention + MLP projections
• Final step reward ≈ 0.41, reward_std ≈ 0.24 (80/80 steps)