MemSearcher-3B

MemSearcher-3B

<p align="center"> 📄 <a href="https://arxiv.org/abs/2511.02805">Paper</a> &nbsp;|&nbsp; 💻 <a href="https://github.com/icip-cas/MemSearcher">Code</a> </p>

Model Summary

MemSearcher-3B is a search agent built on Qwen/Qwen2.5-3B-Instruct and trained with multi-context GRPO, the end-to-end reinforcement-learning method introduced in MemSearcher.

Unlike ReAct-style agents that concatenate the entire interaction history into the context — producing long, noisy inputs and linearly growing compute — MemSearcher maintains a compact, iteratively-updated memory. At every turn the backbone LLM sees only two short inputs, the question and the current memory, then it reasons, decides whether to search, and rewrites the memory to keep only question-relevant information. This keeps the context length nearly constant across turns without sacrificing accuracy.

Evaluation

Exact Match (EM) on seven single- and multi-hop QA benchmarks (Wikipedia-18 corpus):

This card is for the 3B model (highlighted row).

Key Features

• 🧠 Compact memory — keeps only question-relevant information instead of the full history.
• 📏 Near-constant context — token count stays stable across multi-turn interactions, cutting compute/memory.
• 🎯 Multi-context GRPO — propagates trajectory-level advantages to every turn for true end-to-end RL.
• 🔍 Strong QA performance — outperforms strong history-concatenation (ReAct-style) baselines across 7 benchmarks.

How It Works

At turn i the context is just (question, memory_{i-1}). The model:

1. Reasons inside <think>...</think>;
2. Searches when it lacks knowledge (a wikipedia_search tool call), or answers in \boxed{} when the memory suffices;
3. Updates the memory from the retrieved results, retaining only what helps answer the question.

Training uses multi-context GRPO: it samples a group of trajectories, then propagates each trajectory's advantage to all of its per-turn contexts, enabling stable end-to-end optimization of reasoning, search, and memory management.

Quickstart

MemSearcher is an agent that needs a Wikipedia retriever and the multi-turn orchestration loop. The simplest path is the released code:

git clone https://github.com/icip-cas/MemSearcher && cd MemSearcher
# 1) start the wiki-18 retriever (see README "Retriever Serving")
# 2) serve this model with SGLang
MODEL_PATH=yuanqianhao/MemSearcher-3B TP=2 PORT=80 bash launch_server.sh
# 3) run the agentic evaluation
GENERATOR_MODEL=yuanqianhao/MemSearcher-3B DATA_DIR=./data SAVE_DIR=./eval \
  SGL_REMOTE_URL=http://127.0.0.1:80 RETRIEVER_URL=http://127.0.0.1:8000/search bash eval.sh

The model weights load with standard transformers (AutoModelForCausalLM.from_pretrained("yuanqianhao/MemSearcher-3B")), but reproducing the reported numbers requires the retriever + agent loop above.

Training Data

Trained on the NQ + HotpotQA training split — the same data as Search-R1 — re-wrapped into MemSearcher's re_call format. See the code repository for the exact preparation script.

Citation

@article{yuan2025memsearcher,
  title={MemSearcher: Training LLMs to Reason, Search and Manage Memory via End-to-End Reinforcement Learning},
  author={Yuan, Qianhao and Lou, Jie and Li, Zichao and Chen, Jiawei and Lu, Yaojie and Lin, Hongyu and Sun, Le and Zhang, Debing and Han, Xianpei},
  journal={arXiv preprint arXiv:2511.02805},
  year={2025}
}

License

Released under Apache-2.0, following the base model Qwen/Qwen2.5-3B-Instruct.