Downloads · 30 days
0
jhhj25/reef-nccl-ep-qwen3-coder-30b-a3b-lora-nccl-ep-tune-30b
reef-nccl-ep-qwen3-coder-30b-a3b-lora-nccl-ep-tune-30b is a reinforcement learning model from jhhj25. Use it for the reinforcement learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as apache-2.0.
LoRA adapter for Qwen/Qwen3-Coder-30B-A3B-Instruct (MoE) trained with the reef TTTD recipe: an agentic RL loop built on slime with a Megatron-LM training backend (LoRA rank 32, alpha 32, targets q/k/v/o projections).
Downloads · 30 days
0
Access
Public
Updated Sep 5, 2026
Repo size
161 MB
Likes
0
Public
Click a slice to open those files.
.safetensors161 MB · 100%
From the Hugging Face model README
LoRA adapter for Qwen/Qwen3-Coder-30B-A3B-Instruct (MoE) trained with the reef TTTD recipe: an agentic RL loop built on slime with a Megatron-LM training backend (LoRA rank 32, alpha 32, targets q/k/v/o projections).
Task: the agent proposes a JSON configuration for NCCL-based expert-parallel MoE all-to-all communication (dispatch/combine tuning knobs such as queue pairs, chunk size, and SM allocation). Each proposed config is benchmarked by a judge on a multi-node GPU cluster, and the reward is the measured combined dispatch+combine throughput in GB/s. For scale: a hand-tuned expert configuration on the same setup reaches ~193 GB/s total.
| Step | Reward (total GB/s) |
|---|---|
| 0 | 139.82 |
| 1 | 137.63 |
| 2 | 136.28 |
step_0/, step_1/, step_2/ — HF/PEFT-format LoRA adapters exported at each
training step (every intermediate checkpoint of the run).from peft import PeftModel
from transformers import AutoModelForCausalLM
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-Coder-30B-A3B-Instruct")
model = PeftModel.from_pretrained(base, "jhhj25/reef-nccl-ep-qwen3-coder-30b-a3b-lora-nccl-ep-tune-30b", subfolder="step_2")