reef-nccl-ep-qwen3-8b-lora-coevolve-run2

reef-nccl-ep-qwen3-8b-lora-coevolve-run2

LoRA adapters (rank 32, base model Qwen/Qwen3-8B) from a 16-step reinforcement learning run on an expert-parallel collective-communication kernel tuning task. This repository contains all 16 intermediate checkpoints, one per training step, in subfolders step_0 through step_15.

Training setup

• 16 steps of grouped relative-reward RL using the reef TTTD harness for rollouts and slime/Megatron for LoRA training.
• The task asks the model to choose tuning parameters for expert-parallel dispatch/combine collective kernels. The reward is the hardware-measured dispatch+combine bandwidth in GB/s, with a hard correctness gate: samples that fail correctness checks receive no reward.
• LoRA configuration: r=32, alpha=32, dropout 0.0, target modules q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj.

Co-evolution context

This is run 2 of a harness/model co-evolution experiment. Before this run, a 480B model proposed a revised task instruction. The revision was adopted after it beat the incumbent instruction on a fixed checkpoint: mean sample quality went from 138.28 to 176.56 on the same weights. This run then trained on the evolved instruction. The previous run, trained on the original instruction, had per-step mean rewards in the range 126.7 to 164.2.

Per-step mean reward

Best configuration found during this run

qps=2, chunk=256, subputs=16, cb=32, sms=12 reaching 184.74 GB/s total dispatch+combine bandwidth, with correctness 16/16.

Loading

The adapters were saved with a local base-model path, so pass the base model explicitly:

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-8B", torch_dtype="bfloat16")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-8B")
model = PeftModel.from_pretrained(base, "jhhj25/reef-nccl-ep-qwen3-8b-lora-coevolve-run2", subfolder="step_15")

Replace step_15 with any of step_0 ... step_15 to load an earlier checkpoint.