Youtu-LLM-2B-ONNX

<div align="center">

<img src="https://huggingface.co/tencent/Youtu-LLM-2B/resolve/main/assets/youtu-llm-logo.png" alt="Youtu-LLM Logo" height="100px">

📃 License • 💻 Code • 📑 Technical Report • 📊 Benchmarks • 🚀 Getting Started • 💡 Highlights

</div>

🎯 Brief Introduction

Youtu-LLM is a new, small, yet powerful LLM, contains only 1.96B parameters, supports 128k long context, and has native agentic talents. On general evaluations, Youtu-LLM significantly outperforms SOTA LLMs of similar size in terms of Commonsense, STEM, Coding and Long Context capabilities; in agent-related testing, Youtu-LLM surpasses larger-sized leaders and is truly capable of completing multiple end2end agent tasks.

Youtu-LLM has the following features:

• Type: Autoregressive Causal Language Models with Dense MLA
• Release versions: Base and Instruct
• Number of Parameters: 1.96B
• Number of Layers: 32
• Number of Attention Heads (MLA): 16 for Q/K/V
• MLA Rank: 1,536 for Q, 512 for K/V
• MLA Dim: 128 for QK Nope, 64 for QK Rope, and 128 for V
• Context Length: 131,072
• Vocabulary Size: 128,256

🤗 Model Download

📰 News

• [2026.01.07] You can now fine-tuning Youtu-LLM with ModelScope.
• [2026.01.04] You can now fine-tuning Youtu-LLM with LlamaFactory.

<a id="benchmarks"></a>

📊 Performance Comparisons

Instruct Model

<img src="https://huggingface.co/tencent/Youtu-LLM-2B/resolve/main/assets/general_agentic_instruct.png" alt="Comparison between Youtu-LLM-2B and baselines" height="260px">

General Benchmarks

Agentic Benchmarks

<a id="quickstart"></a>

🚀 Quick Start

This guide will help you quickly deploy and invoke the Youtu-LLM-2B model. This model supports "Reasoning Mode", enabling it to generate higher-quality responses through Chain of Thought (CoT).

ONNXRuntime

import json
from transformers import AutoTokenizer
import onnxruntime
import numpy as np

# 1. Load config, processor, and model
## git clone https://huggingface.co/onnx-community/Youtu-LLM-2B-ONNX
model_folder = "/path/to/model/Youtu-LLM-2B-ONNX"
with open(f"{model_folder}/config.json", "r") as f:
  config = json.load(f)
tokenizer = AutoTokenizer.from_pretrained(model_folder)
decoder_session = onnxruntime.InferenceSession(f"{model_folder}/onnx/model.onnx")

## Set config values
num_key_value_heads = config['num_key_value_heads']
head_dim = config['qk_head_dim']
num_hidden_layers = config['num_hidden_layers']
eos_token_id = config['eos_token_id']

# 2. Prepare inputs
messages = [
  {"role": "user", "content": "What is the capital of France?"},
]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="np")
input_ids = inputs['input_ids']
attention_mask = inputs['attention_mask']
batch_size = input_ids.shape[0]
past_key_values = {
  f'past_key_values.{layer}.{kv}': np.zeros([batch_size, num_key_value_heads, 0, head_dim], dtype=np.float32)
  for layer in range(num_hidden_layers)
  for kv in ('key', 'value')
}
position_ids = np.tile(np.arange(0, input_ids.shape[-1]), (batch_size, 1))

# 3. Generation loop
max_new_tokens = 1024
generated_tokens = np.array([[]], dtype=np.int64)
for i in range(max_new_tokens):
  logits, *present_key_values = decoder_session.run(None, dict(
    input_ids=input_ids,
    attention_mask=attention_mask,
    position_ids=position_ids,
    **past_key_values,
  ))

  ## Update values for next generation loop
  input_ids = logits[:, -1].argmax(-1, keepdims=True)
  attention_mask = np.concatenate([attention_mask, np.ones_like(input_ids, dtype=np.int64)], axis=-1)
  position_ids = position_ids[:, -1:] + 1
  for j, key in enumerate(past_key_values):
    past_key_values[key] = present_key_values[j]

  generated_tokens = np.concatenate([generated_tokens, input_ids], axis=-1)
  if np.isin(input_ids, eos_token_id).any():
    break

  ## (Optional) Streaming
  print(tokenizer.decode(input_ids[0]), end='', flush=True)
print()

# 4. Output result
print(tokenizer.batch_decode(generated_tokens, skip_special_tokens=True)[0])

Key Configuration Details

Reasoning Mode Toggle

Controlled via the enable_thinking parameter in the apply_chat_template method:

• True (Recommended Default): Activates Chain of Thought; ideal for complex logic and reasoning tasks.
• False: Outputs results directly; faster response time, suitable for simple conversations.

Recommended Decoding Parameters

Depending on your use case, we suggest adjusting the following hyperparameters for optimal generation:

Tip: When using Reasoning Mode, a higher temperature helps the model perform deeper, more divergent thinking.

<a id="highlights"></a>

💡 Highlights

Showcase of General Tasks

Showcase of Specialized Tasks

Note: For specialized tasks, in-domain post-training is further applied.

📚 Citation

If you find our work useful in your research, please consider citing the following paper:

@article{youtu-llm,
  title={Youtu-LLM: Unlocking the Native Agentic Potential for Lightweight Large Language Models},
  author={Tencent Youtu Lab},
  year={2025},
  eprint={2512.24618},
  archivePrefix={arXiv},
  primaryClass={cs.CL},
  url={https://arxiv.org/abs/2512.24618}, 
}