Downloads · 30 days
0
jhhj25/reef-nccl-ep-qwen3-8b-lora-coevolve-m4
reef-nccl-ep-qwen3-8b-lora-coevolve-m4 is a reinforcement learning model from jhhj25. Use it for the reinforcement learning task on the model card, and read the license before you ship it in a product. It is set up for peft.
LoRA adapters (r32) from a reef TTTD (test-time training on discovery) run, phase 4 of a harness/model co-evolution loop. Base model: Qwen/Qwen3-8B. Task: propose configurations for a systems-tuning discovery problem;…
Downloads · 30 days
0
Access
Public
Updated Sep 6, 2026
Repo size
2.8 GB
Likes
0
Public
Click a slice to open those files.
.safetensors2.8 GB · 100%
From the Hugging Face model README
LoRA adapters (r32) from a reef TTTD (test-time training on discovery) run, phase 4 of a harness/model co-evolution loop. Base model: Qwen/Qwen3-8B. Task: propose configurations for a systems-tuning discovery problem; reward is a measured throughput score returned by an external judge. This phase trained on an instruction adopted after gating four candidate instructions against a frozen checkpoint (best-of-N criterion).
All 16 intermediate training steps are included as step_0 ... step_15.
| step | mean reward |
|---|---|
| 0 | 175.52 |
| 1 | 169.30 |
| 2 | 173.33 |
| 3 | 183.72 |
| 4 | 175.67 |
| 5 | 184.74 |
| 6 | 174.51 |
| 7 | 184.74 |
| 8 | 174.90 |
| 9 | 184.74 |
| 10 | 172.72 |
| 11 | 184.74 |
| 12 | 174.27 |
| 13 | 184.74 |
| 14 | 172.71 |
| 15 | 184.74 |
Episode best-of-run reward: 185.76. Note: from step 5 the run alternates between a dominant sampled configuration (184.74 every odd step) and exploratory rollouts that score lower; the dominant mode moved up from the previous phase (176.56 -> 184.74) after the instruction adoption.
Load any step with PEFT on top of Qwen/Qwen3-8B:
from peft import PeftModel
from transformers import AutoModelForCausalLM
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-8B")
model = PeftModel.from_pretrained(base, "jhhj25/reef-nccl-ep-qwen3-8b-lora-coevolve-m4", subfolder="step_15")