Downloads · 30 days
0
jhhj25/reef-nccl-ep-qwen3-8b-lora-discovery-16step
reef-nccl-ep-qwen3-8b-lora-discovery-16step is a reinforcement learning model from jhhj25. Use it for the reinforcement learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as apache-2.0.
This repository holds the surviving checkpoint from a 16-step reinforcement-learning training run in which a Qwen3-8B model learned to propose configurations for an expert-parallel collective-communication kernel (dis…
Downloads · 30 days
0
Access
Public
Updated Sep 6, 2026
Repo size
175 MB
Likes
0
Public
Click a slice to open those files.
.safetensors175 MB · 100%
From the Hugging Face model README
This repository holds the surviving checkpoint from a 16-step reinforcement-learning training run in which a Qwen3-8B model learned to propose configurations for an expert-parallel collective-communication kernel (dispatch + combine). The reward for each proposed configuration was its bandwidth measured on real hardware (two nodes, 8x H200 GPUs each), gated on a 16/16-pass correctness test; configurations that failed correctness received no reward.
Only the step 8 adapter survived a storage incident: the training work directory was deleted by a follow-on job before the checkpoints were archived. The step-8 adapter was recovered from the memory of a still-running inference server that had the file mapped, and validates as a complete PEFT LoRA safetensors file (504 tensors, all 36 layers, shapes consistent with Qwen3-8B). The other 15 checkpoints are lost. The full training curve is reproduced below from the run logs.
step_8/ — PEFT LoRA adapter, rank 32, alpha 32, dropout 0, target modules
q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj.The adapter_config.json was reconstructed from the training recipe configuration
(the original file was lost with the directory); the safetensors weights are
byte-identical across two independent memory dumps (sha256
93d717a6d120d1a12faa5dbbfa34eb47e34e1b77c97bc846471b8246f89da713).
| step | mean reward | step | mean reward | |
|---|---|---|---|---|
| 0 | 130.04 | 8 | 154.04 | |
| 1 | 139.93 | 9 | 163.15 | |
| 2 | 140.11 | 10 | 147.21 | |
| 3 | 150.09 | 11 | 164.15 | |
| 4 | 127.15 | 12 | 138.14 | |
| 5 | 147.80 | 13 | 152.92 | |
| 6 | 145.38 | 14 | 126.67 | |
| 7 | 158.84 | 15 | 155.32 |
Best single episode: 175.69 GB/s (config q2 c256 sp32 cb64 sms12, correctness 16/16). For scale: a 480B-parameter model prompted zero-shot on the same task and scored by the same judge reached 147.46 GB/s, so the best configuration discovered by the trained 8B model exceeds the zero-shot proposal of the much larger model.
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-8B", torch_dtype="bfloat16")
model = PeftModel.from_pretrained(
base,
"jhhj25/reef-nccl-ep-qwen3-8b-lora-discovery-16step",
subfolder="step_8",
)
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-8B")