SESA-8B-search

SESA-8B: Self-Evolving Search Agent

<p align="center"> <img src="framework.png" alt="SESA framework overview" width="100%"> </p>

SESA-8B is a search-augmented reasoning agent built on Qwen3-8B and trained with a self-play reinforcement-learning framework. A Proposer and a Solver co-evolve: the Proposer invents hard multi-hop search questions, the Solver answers them with tool calls, and the Solver's own failures are distilled into reusable skills that make the agent progressively stronger at multi-hop retrieval and reasoning.

---

Method

SESA trains a search agent without a fixed, human-curated question set. Two roles improve one another in a closed loop:

1. Proposer Rollout — Given only a target answer, the Proposer actively searches, digs up implicit bridging facts, and composes a question that requires several sequential searches to solve. Trivial questions (answerable in one hop or from general knowledge) are validated away.
2. Solver Rollout — The extracted question is handed to the Solver, which retrieves relevant skills, reasons step by step, plans a search strategy, issues tool calls, and produces an answer.
3. Failure-to-Skill Evolution — Failed Solver rollouts are collected and abstracted into structured Skill Cards (category, pattern, distinction, triggers, query templates) stored in a growing Skill Bank.

The loop is virtuous: failures become skills → the Solver improves → the Proposer must pose harder questions, driving continual self-improvement.

• Base model: Qwen3-8B
• Training: GRPO self-play with an online skill bank
• Precision: bfloat16
• Intended use: multi-hop question answering with a search/retrieval tool

---

Evaluation

Semantic-correctness (LLM-as-judge) accuracy on seven single-hop and multi-hop QA benchmarks (500 samples each; Bamboogle 125):

SESA-8B improves the average by +7.0 points over the Qwen3-8B base, with the largest gains on multi-hop sets (HotpotQA +9.8, 2Wiki +6.6, MuSiQue +6.0).

Evaluation uses a search tool over a local dense retriever (Wikipedia-style corpus). Absolute numbers depend on the retrieval backend and judge model.

---

Usage

The model uses a multi-turn tool-calling format with a search tool. To reproduce the agentic setup, pair it with a retrieval service (see the project repository). For plain inference:

from modelscope import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "fzh463626021/SESA-8B-search"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map="auto")

messages = [{"role": "user", "content": "Give me a short introduction to large language model."}]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
outputs = model.generate(inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True))

For the full search-agent pipeline (retrieval server, tool config, multi-turn rollout), see the project code.

---

Limitations

• The agent is trained for tool-augmented QA; without a search tool its multi-hop accuracy drops.
• Retrieval quality bounds end-task performance — a weak or out-of-domain corpus will lower scores.
• Like all LLMs, it can hallucinate; verify critical facts.

---

Citation

@misc{fu2026selfplaymeetsskillevolution,
      title={Self-Play Meets Skill Evolution: Self-Evolving Search Agents that Pose, Solve, and Remember}, 
      author={Zenghuang Fu and Zhaoyang Li and Qiuyuan Ai and Haoyu Wu and Minghui Wu and Chenxu Zhao and Ante Wang and Guannan He and Changwei Wang},
      year={2026},
      eprint={2607.29468},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2607.29468}, 
}

License

Released under the Apache 2.0 license, consistent with the Qwen3-8B base model.