Downloads · 30 days
0
jhhj25/reef-tttd-qwen3-8b-lora-tttd-smoke
reef-tttd-qwen3-8b-lora-tttd-smoke is a reinforcement learning model from jhhj25. Use it for the reinforcement learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as apache-2.0.
LoRA adapter for Qwen/Qwen3-8B trained with the reef TTTD (test-time task discovery) recipe: an agentic RL loop built on slime with a Megatron-LM training backend (LoRA rank 32, alpha 32, targets q/k/v/o/gate/up/down…
Downloads · 30 days
0
Access
Public
Updated Sep 5, 2026
Repo size
18.6 GB
Likes
0
Public
Click a slice to open those files.
.distcp17.4 GB · 94%
From the Hugging Face model README
LoRA adapter for Qwen/Qwen3-8B trained with the reef TTTD (test-time task discovery) recipe: an agentic RL loop built on slime with a Megatron-LM training backend (LoRA rank 32, alpha 32, targets q/k/v/o/gate/up/down projections, tensor parallel 2).
This was a single-step smoke run of the full RL loop (rollout agents in sandboxed environments, a programmatic judge, and one training commit) on a small graph-theory task (erdos minimum-overlap). Episode reward at step 0 was 0.0; the goal of the run was end-to-end loop verification, not task performance.
step_0/ — HF/PEFT-format LoRA adapter exported at training step 0.megatron/iter_0000000/ — raw Megatron distributed checkpoint (torch distcp shards,
includes optimizer/model state as saved by the trainer).megatron/reef_adapter_slots/ — reef per-slot adapter tensors (rank_*.pt) paired
with the distcp checkpoint.from peft import PeftModel
from transformers import AutoModelForCausalLM
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-8B")
model = PeftModel.from_pretrained(base, "jhhj25/reef-tttd-qwen3-8b-lora-tttd-smoke", subfolder="step_0")