MUA-RL-8B

Introduction

Model Description

We present MUA-RL Model, a multi-turn user-interacting agent reinforcement learning model designed for agentic tool use. This model is specifically designed for multi-turn conversation scenarios where agents need to maintain context across conversations while effectively utilizing tools to complete complex tasks. MUA-RL is the first framework to integrate LLM-simulated users into the reinforcement learning loop for agentic tool use, enabling autonomous learning of models to communicate with users efficiently and use various tools to solve practical problems in dynamic multi-turn interactions.

Performance

MUA-RL achieves competitive performance across multiple multi-turn tool-using benchmarks:

The model outperforms or matches the performance of larger open-source models such as DeepSeek-V3-0324 and Qwen3-235B-A22B in non-thinking settings.

Training Details

Architecture

• Model Size: 8B parameters
• Train Context Length: 32K tokens

Training Process

• Reinforcement Learning: Group Relative Policy Optimization (GRPO)
• User Simulation: LLM-simulated users integrated into RL loop (GPT-4o-2024-11-20 as user)
• Environment Management: Environment creation for each rollout
• Tool Integration: Seamless tool calling and response handling

Citation

If you use MUA-RL in your research, please cite our paper:

@misc{zhao2025mua,
  title={MUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for Agentic Tool Use},
  author={Weikang Zhao and Xili Wang and Chengdi Ma and Lingbin Kong and Zhaohua Yang and Mingxiang Tuo and Xiaowei Shi and Yitao Zhai and Xunliang Cai},
  year={2025},
  eprint={2508.18669},
  archivePrefix={arXiv},
  primaryClass={cs.AI},
  url={https://arxiv.org/abs/2508.18669}
}