dt-4bT-r2e-ep1

wenyii/dt-4bT-r2e-ep1

Multi-turn agent SFT on Qwen3-4B-Thinking-2507 — r2e arm, checkpoint at training step ep1.

Part of an ICLR-target analysis study on cross-domain transfer of multi-turn agent SFT (Qwen3-4B-Thinking, dual-terminator recipe). Sister checkpoints in the same family:

Full step curves (s50/s100/s150/s200/ep1/ep2) available on request.

Training recipe ("dt" = dual-terminator)

• Base: Qwen/Qwen3-4B-Thinking-2507
• Data: 3500 multi-turn trajectories per arm
  • r2e: code-agent trajectories (R2E / SWE-Gym style, ending in <tool_call>{finish})
  • search: search-agent trajectories (OpenSeeker + NQ-style, ending in Final answer: X)
  • mixed: 50/50 shuffle of r2e + search
• Recipe diff vs. v5 (one-line edit): for the r2e arm, append Final answer: <pr_description-headline, ≤200 chars> epilogue AFTER the standard <tool_call>{finish}. This dual-terminates the trajectory so a downstream literal-EM evaluator (Search-R1) can still extract the final answer.
• Optimizer: LR 1e-5, cosine schedule, warmup 0.03, weight decay 0, grad clip 1.0, batch 16, FSDP1 + Liger + flash-attn (bf16).
• Saves: steps 50, 100, 150, 200, 218 (=ep1), 436 (=ep2 for search/mixed; r2e capped at ep1).

How to use

from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("wenyii/dt-4bT-r2e-ep1")
mdl = AutoModelForCausalLM.from_pretrained("wenyii/dt-4bT-r2e-ep1", torch_dtype="bfloat16")

Or with vLLM:

python -m vllm.entrypoints.openai.api_server \
    --model wenyii/dt-4bT-r2e-ep1 --port 8000 --tensor-parallel-size 1

For full multi-turn agent inference (retriever + tool-call parsing + answer cascade extractor), see the reference SR1 7-bench eval pipeline distributed alongside the paper.

Headline numbers (Search-R1 7-bench, N=500 per bench, mml=16384)

dt-r2e-s100 reaches EM 0.344 / F1 0.430 on SR1 7-bench, vs Qwen3-4B-Thinking-2507 base + Final answer: prompt nudge anchor at EM 0.302 / F1 0.389. See the companion paper for the full 7-bench / per-arm / multi-seed analysis.

Citation

@inproceedings{wenyimo2027dt4b,
  title={...},
  author={Mo, Wenyi and ...},
  booktitle={ICLR 2027},
  year={2027}
}