Downloads · 30 days
0
jhhj25/reef-nccl-ep-qwen3-8b-lora-coevolve-run5
reef-nccl-ep-qwen3-8b-lora-coevolve-run5 is a reinforcement learning model from jhhj25. Use it for the reinforcement learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as apache-2.0.
Run 5 of a harness/model co-evolution loop. A large (480B) proposer model revises the task instruction given to a Qwen3-8B policy; a revised instruction is adopted only when a fixed-checkpoint gate shows it improves t…
Downloads · 30 days
0
Access
Public
Updated Sep 6, 2026
Repo size
2.8 GB
Likes
0
Public
Click a slice to open those files.
.safetensors2.8 GB · 100%
From the Hugging Face model README
Run 5 of a harness/model co-evolution loop. A large (480B) proposer model revises the
task instruction given to a Qwen3-8B policy; a revised instruction is adopted only when
a fixed-checkpoint gate shows it improves the policy's best result. The policy is then
trained for 16 steps of grouped relative-reward RL (reef TTTD + slime/Megatron LoRA,
4 groups x 8 rollouts per step, sampling temperature 1.15). This repository contains
all 16 intermediate LoRA adapter checkpoints from run 5 (step_0 ... step_15).
Expert-parallel collective-communication kernel configuration tuning. The policy proposes a kernel config; the reward is hardware-measured dispatch+combine bandwidth (GB/s) with a hard correctness gate (configs that fail correctness receive no reward).
Run 5 trained on the third adopted instruction. The proposer's revision led the policy into the human-optimal region: per-step reward means ranged 190.3-193.0, and the policy samples the 192.99 GB/s config
qps=2 chunk=256 subputs=6 cb=32 sms=11
as its attractor. This is the human-tuned optimum for the task, reproduced and internalized by the trained model. Across the three instruction adoptions the policy's attractor moved 176.56 -> 184.74 -> 192.99 GB/s.
Configs first sampled by the policy in earlier runs and verified on hardware (correctness 16/16):
| config | bandwidth (GB/s) |
|---|---|
| qps=2 chunk=256 subputs=16 cb=32 sms=12 | 184.74 |
| qps=2 chunk=256 subputs=16 cb=64 sms=12 | 185.76 |
| human-tuned reference | 192.99 |
Each step_N directory is a PEFT LoRA adapter (rank 32, alpha 32, dropout 0.0,
targets: q/k/v/o_proj + gate/up/down_proj) saved after RL step N. Base model:
Qwen3-8B.
Note: adapter_config.json records the training-time base model path work/model;
pass the base model explicitly.
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-8B", torch_dtype="bfloat16")
model = PeftModel.from_pretrained(
base, "jhhj25/reef-nccl-ep-qwen3-8b-lora-coevolve-run5", subfolder="step_15"
)
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-8B")