Downloads · 30 days
0
jhhj25/reef-nccl-ep-qwen3-8b-lora-coevolve-run4
reef-nccl-ep-qwen3-8b-lora-coevolve-run4 is a reinforcement learning model from jhhj25. Use it for the reinforcement learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as apache-2.0.
Run 4 of a harness/model co-evolution loop. Before this run, a 480B proposer model revised the task instruction; the revision was adopted after a fixed-checkpoint gate showed it breaks the policy's mode collapse. The…
Downloads · 30 days
0
Access
Public
Updated Sep 6, 2026
Repo size
2.8 GB
Likes
0
Public
Click a slice to open those files.
.safetensors2.8 GB · 100%
From the Hugging Face model README
Run 4 of a harness/model co-evolution loop. Before this run, a 480B proposer model revised the task instruction; the revision was adopted after a fixed-checkpoint gate showed it breaks the policy's mode collapse. The policy (Qwen3-8B with a rank-32 LoRA adapter) was then trained for 16 steps of grouped relative-reward RL (reef TTTD with slime/Megatron LoRA training, 4 groups x 8 rollouts per step, sampling temperature 1.15).
The policy proposes configurations for an expert-parallel collective-communication kernel (dispatch and combine phases). The reward is the hardware-measured dispatch+combine bandwidth in GB/s, behind a hard correctness gate: configurations that fail any correctness check receive no reward.
The training attractor moved from 176.56 GB/s (runs 2 and 3) to 184.74 GB/s in this run: by the end of run 4, even-numbered steps sample the 184.74 GB/s configuration reliably.
Best configurations discovered across runs 2-4 (both pass correctness 16/16):
| qps | chunk | subputs | cb | sms | bandwidth (GB/s) |
|---|---|---|---|---|---|
| 2 | 256 | 16 | 32 | 12 | 184.74 |
| 2 | 256 | 16 | 64 | 12 | 185.76 |
The human-tuned reference configuration reaches 192.99 GB/s.
step_0 through step_15: one PEFT LoRA adapter per RL training step
(rank 32, alpha 32, all attention and MLP projections of Qwen3-8B).
step_15 is the final adapter of the run.
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-8B", torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(base, "jhhj25/reef-nccl-ep-qwen3-8b-lora-coevolve-run4", subfolder="step_15")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-8B")
Note: adapter_config.json records the local training path work/model as
the base model; load against Qwen/Qwen3-8B as shown above.