Nanbeige4-3B-Thinking-2510

<div align="center"> <img src="figures/nbg.png" width="220" alt="Nanbeige Logo"> </div>

<span id="Pre-Training">1. Introduction</span>

Nanbeige4-3B-Thinking is a 3B-parameter reasoning model within the fourth-generation Nanbeige LLM family. It showcases that even compact models can achieve advanced reasoning abilities through continuous enhancements in data quality and training methodologies. To support research and technological advancement in the open-source community, we have open-sourced the Nanbeige4-3B-Thinking model together with its technical methodology.

<div align="center"> <img src="figures/performance.png"> </div>

<span id="Pre-Training">2. Model Summary</span>

Pre-Training<br>

• We constructed a comprehensive 23T-tokens training corpus from web texts, books, code, and papers, meticulously filtered through a hybrid strategy of tagging-based scoring and retrieval-based recalling. This foundation was then augmented with knowledge-dense and reasoning-intensive synthetic data, including Q&A pairs, textbooks, and Long-COTs, which significantly benefited the downstream task performance.

<!-- * For training Data, after extensive data curation and crawling, we designed and employed a data filtering strategy that combines tagging-based scoring with retrieval-based recalling to filter for high-quality data. We ultimately constructed a **23T-token** training corpus, comprising web pages, books, code, papers, and more. Besides realistic data, we also incorporate synthetic data with high knowledge density and reasoning density, such as QAs, Textbooks, and Long-COTs, which significantly benefited the downstream task performance. -->

• We designed an innovative FG-WSD (Fine-Grained Warmup-Stable-Decay) training scheduler, meticulously refining the conventional WSD approach. This scheduler was implemented with a fine-grained, quality-progressive data curriculum, dividing the Stable stage into multiple phases with progressively improved data mixtures. Compared to the vanilla WSD, our method achieved notable performance gains. During the Decay stage, we increased the proportion of math, code, synthetic QA, and synthetic Long-COT data to further enhance reasoning capabilities.

<!-- * For training recipe, we innovatively proposed the **FG-WSD** (Fine-Grained Warmup-Stable-Decay) scheduler, as an improvement upon the WSD (Warm-Stable-Decay). In the Stable stage, we divided 19T tokens into multiple fine-grained phases, with later phases using an overall higher-quality data mix. Compared to the vanilla WSD scheduler, FG-WSD achieved promising benefits. In the Decay stage, we use 4T tokens, in which the proportion of math, code, synthetic QA, and synthetic Long-COT is increased to enhance the model's reasoning capabilities. -->

Post-Training<br>

• SFT phase. We constructed a collection of over 30 million high-quality Long Chain-of-Thought (Long-CoT) samples to support multi-stage curriculum learning. By integrating both rule-based and model-based verification methods, we not only ensured sample accuracy but also enhanced the comprehensiveness and instructional value of each training example compared to alternative candidates. This rich diversity in instructions and high response quality equipped the model to achieve outstanding performance across a variety of benchmarks. <br>

• Distill. Following SFT, we employed the Nanbeige flagship reasoning model as the teacher model to distill the Nanbeige4-3B-Thinking model, and further enhanced the performance. We observed that on-policy distillation provides greater benefits for mathematical reasoning tasks, while off-policy distillation is more effective for general tasks such as human-preference alignment.<br>

• RL phase. We then advanced to a multi-stage, on-policy reinforcement learning phase. This approach leverages verifiable rewards to enhance reasoning capability and a preference reward model to improve alignment, utilizing a carefully filtered blend of real-world and synthetic data calibrated for appropriate difficulty.<br><br><br>

<span id="Performance">3. Model Performance</span>

For model performance comparison, we benchmark our model against recent reasoning LLMs from the Qwen3 series. All models are evaluated under identical configurations to ensure fairness. The results show that our model outperforms the baselines across a range of mainstream benchmarks, including math, science, creative writing, tool use, and human preference alignment.

<span id="Inference">4. Quickstart</span>

For the chat scenario:

from transformers import AutoModelForCausalLM, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained(
  'Nanbeige/Nanbeige4-3B-Thinking-2510',
  use_fast=False,
  trust_remote_code=True
)
model = AutoModelForCausalLM.from_pretrained(
  'Nanbeige/Nanbeige4-3B-Thinking-2510',
  torch_dtype='auto',
  device_map='auto',
  trust_remote_code=True
)
messages = [
  {'role': 'user', 'content': 'Which number is bigger, 9.11 or 9.8?'}
]
prompt = tokenizer.apply_chat_template(
  messages,
  add_generation_prompt=True,
  tokenize=False
)
input_ids = tokenizer(prompt, add_special_tokens=False, return_tensors='pt').input_ids
output_ids = model.generate(input_ids.to('cuda'), eos_token_id=166101)
resp = tokenizer.decode(output_ids[0][len(input_ids[0]):], skip_special_tokens=True)
print(resp)

For the tool use scenario:

from transformers import AutoModelForCausalLM, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained(
  'Nanbeige/Nanbeige4-3B-Thinking-2510',
  use_fast=False,
  trust_remote_code=True
)
model = AutoModelForCausalLM.from_pretrained(
  'Nanbeige/Nanbeige4-3B-Thinking-2510',
  torch_dtype='auto',
  device_map='auto',
  trust_remote_code=True
)
messages = [
    {'role': 'user',  'content': 'Help me check the weather in Beijing now'}
]
tools = [{'type': 'function',
  'function': {'name': 'SearchWeather',
   'description': 'Find out current weather in a certain place on a certain day.',
   'parameters': {'type': 'dict',
    'properties': {'location': {'type': 'string',
      'description': 'A city in china.'},
    'required': ['location']}}}}]
prompt = tokenizer.apply_chat_template(
  messages,
  tools,
  add_generation_prompt=True,
  tokenize=False
)
input_ids = tokenizer(prompt, add_special_tokens=False, return_tensors='pt').input_ids
output_ids = model.generate(input_ids.to('cuda'), eos_token_id=166101)
resp = tokenizer.decode(output_ids[0][len(input_ids[0]):], skip_special_tokens=True)
print(resp)

<span id="Limitations">5. Limitations</span>

While we place great emphasis on the safety of the model during the training process, striving to ensure that its outputs align with ethical and legal requirements, it may not completely avoid generating unexpected outputs due to the model's size and probabilistic nature. These outputs may include harmful content such as bias or discrimination. Please don't propagate such content. We do not assume any responsibility for the consequences resulting from the dissemination of inappropriate information. <br>

<span id="Limitations">6. Citation</span>

If you find our model useful or want to use it in your projects, please kindly cite this Huggingface project. <br>

<span id="Limitations">7. Contact</span>

If you have any questions, please raise an issue or contact us at [email protected]. <br>