qwen3-8b-rethink-sft

Qwen3-8B-Rethink-SFT

Qwen3-8B fine-tuned to voluntarily invoke a rethink helper sub-agent during AppWorld agent tasks, by distilling trajectories from Qwen3.6-27B.

This is the SFT warm-up checkpoint used as the initial policy for AgentEvolver GRPO training. See the experiment repo: junjie-meng/agentevolver_rethink.

What it does

When solving AppWorld tasks (multi-step Python REPL interaction), the model can output a line of the form

RETHINK: n/question

to ask a helper sub-agent to review the last n trajectory steps and answer a concrete question, then continue solving with the advice. Base Qwen3-8B never produces this pattern under prompting alone (0/300+ calls in our tests); this checkpoint triggers it voluntarily on ~100% of tasks.

Training

• Data: 184 call-level samples from 48 successful Qwen3.6-27B AppWorld trajectories containing voluntary RETHINK usage (both the main-agent calls and the helper-side advice calls). Dataset ships with the experiment repo.
• Full-parameter SFT, loss on target tokens only, rendering byte-identical to serving (Qwen3 non-thinking, empty <think> prefix).
• 2 epochs, lr 5e-5, cosine, global batch 16, bf16, 8×H20.

Evaluation snapshot (AppWorld)

Note: SFT alone transfers the behaviour, not capability — expect train-split contamination on the 48 source tasks and a small generalization dip vs base; capability comes from the subsequent RL stage.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
m = AutoModelForCausalLM.from_pretrained("JunjieMeng/qwen3-8b-rethink-sft", torch_dtype="bfloat16")
t = AutoTokenizer.from_pretrained("JunjieMeng/qwen3-8b-rethink-sft")

Intended for research on agent self-reflection / help-seeking behaviour; inherits Qwen3-8B's license (Apache-2.0).