Youtu-LLM-2B-Base

<div align="center">

<img src="assets/youtu-llm-logo.png" alt="Youtu-LLM Logo" height="100px">

📃 License • 💻 Code • 📑 Technical Report • 📊 Benchmarks

</div>

🎯 Brief Introduction

Youtu-LLM is a new, small, yet powerful LLM, contains only 1.96B parameters, supports 128k long context, and has native agentic talents. On general evaluations, Youtu-LLM significantly outperforms SOTA LLMs of similar size in terms of Commonsense, STEM, Coding and Long Context capabilities; in agent-related testing, Youtu-LLM surpasses larger-sized leaders and is truly capable of completing multiple end2end agent tasks.

Youtu-LLM has the following features:

• Type: Autoregressive Causal Language Models with Dense MLA
• Release versions: Base and Instruct
• Number of Parameters: 1.96B
• Number of Layers: 32
• Number of Attention Heads (MLA): 16 for Q/K/V
• MLA Rank: 1,536 for Q, 512 for K/V
• MLA Dim: 128 for QK Nope, 64 for QK Rope, and 128 for V
• Context Length: 131,072
• Vocabulary Size: 128,256

🤗 Model Download

📰 News

• [2026.01.28] You can now directly use Youtu-LLM with Transformers>=5.1.0.
• [2026.01.07] You can now fine-tuning Youtu-LLM with ModelScope.
• [2026.01.04] You can now fine-tuning Youtu-LLM with LlamaFactory.

Note: If you wish to use Youtu-LLM-2B-Base based on earlier versions of transformers (>=4.56.0,<=4.57.1), please make sure to download the model repository before this commit.

<a id="benchmarks"></a>

📊 Performance Comparisons

Base Model

<img src="assets/general_agentic_base.png" alt="Comparison between Youtu-LLM-2B-Base and baselines" height="260px">

General Benchmarks

Agentic Benchmarks

We takes APTBench for evaluating the agentic capabilities of base model.

📚 Citation

If you find our work useful in your research, please consider citing the following paper:

@article{youtu-llm,
  title={Youtu-LLM: Unlocking the Native Agentic Potential for Lightweight Large Language Models},
  author={Tencent Youtu Lab},
  year={2025},
  eprint={2512.24618},
  archivePrefix={arXiv},
  primaryClass={cs.CL},
  url={https://arxiv.org/abs/2512.24618}, 
}