TinyR1-32B-Preview

Model Name: Tiny-R1-32B-Preview

<span style="color:red">!!! Important Release !!!</span>

<span style="color:red">The official version of TinyR1-32B has been released. Please visit TinyR1-32B</span>

Update(April 16, 2025)

We have officially open-sourced the <a href="https://huggingface.co/datasets/qihoo360/TinyR1-32B-Preview-datasets"><b>training dataset</b></a>, as well as the full <a href="https://github.com/Qihoo360/TinyR1-32B-Preview"><b>training and evaluation pipeline</b></a> .

Update(March 7, 2025)

We have uploaded the technical report. <a href="https://arxiv.org/abs/2503.04872"><b>Paper Link</b>👁️</a>

Introduction

We introduce our first-generation reasoning model, Tiny-R1-32B-Preview, which outperforms the 70B model Deepseek-R1-Distill-Llama-70B and nearly matches the full R1 model in math.

We applied supervised fine-tuning (SFT) to Deepseek-R1-Distill-Qwen-32B across three target domains—Mathematics, Code, and Science — using the 360-LLaMA-Factory training framework to produce three domain-specific models. We used questions from open-source data as seeds. Meanwhile, responses for mathematics, coding, and science tasks were generated by R1, creating specialized models for each domain. Building on this, we leveraged the Mergekit tool from the Arcee team to combine multiple models, creating Tiny-R1-32B-Preview, which demonstrates strong overall performance.

Note: We have fixed a new tokenizer config bug that existed before March 3, 2025, at 20:50 Beijing Time (UTC+8). Refer to Hotfix.

Evaluation

All scores are reported as pass@1. For AIME 2024, we sample 16 responses, and for GPQA-Diamond, we sample 4 responses, both using average overall accuracy for stable evaluation.

We merged the models trained separately in three directions into a single model. Below are the comparison results.

Quickstart

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "qihoo360/TinyR1-32B-Preview"

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_name)

prompt = "Please reason step by step, and put your final answer within \\boxed{}. Solve the integral:  \[I = \int \frac{x^2}{(x+1)^3} \,dx\]"
messages = [
    {"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

generated_ids = model.generate(
    **model_inputs,
    max_new_tokens=4000
)
generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]

response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]

print(response)

Data

1. Math

58.3k CoT trajectories from open-r1/OpenR1-Math-220k, default subset

2. Coding

19k CoT trajectories open-thoughts/OpenThoughts-114k, coding subset

3. Science

8.6k CoT trajectories:

• 2.7k CoT trajectories from simplescaling/data_ablation_full59K, science and health science subset
• 4.9k CoT trajectories from open-thoughts/OpenThoughts-114k, science subset
• 1.0k CoT trajectories from simplescaling/s1K-1.1, all

Open Source Plan

We will publish a technical report as soon as possible and open-source our training and evaluation code, selected training data, and evaluation logs. Having benefited immensely from the open-source community, we are committed to giving back in every way we can.

Caveats

TinyR1-32B-Preview is an experimental research model designed to advance AI reasoning capabilities. As a preview release, it has demonstrated higher evaluation scores on some benchmarks but is not intended for general user applications. Key limitations include:

1. Incorrect parameter configurations may result in repetitive output loops, similar to R1. We recommend setting the temperature to 0.6 and top-p to 0.95, in line with R1's configuration.
2. The model currently omits the <think> token, which indicates the reasoning start, and only outputs the </think> token to signal the end of the thinking process. This will be addressed soon in the next version.
3. The model may generate overly lengthy reasoning chains for simple queries. We are working on improvements.
4. Benchmark coverage is limited. We encourage users to evaluate the model on additional benchmarks, and we will continue to expand our benchmark results.
5. The model requires enhanced safety measures to ensure reliable and secure performance.

Hotfix (March 3, 2025)

On March 3, 2025, at 20:50 Beijing Time (UTC+8), we update our tokenizer. Users who downloaded our model prior to this update are advised to re-download the tokenizer-related configuration files (tokenizer.json, tokenizer_config.json, config.json and special_tokens_map.json). Our internal testing has verified that this update resolves the following issues reported by users:

1. Output repetition.
2. Degradation in benchmark performance.
3. Generation of token IDs exceeding the vocabulary range. We appreciate your feedback and encourage you to report any further issues. Additionally, we are actively working on the technical report and consolidating relevant code and data.

Contributors

360 Team: Lin Sun, Guangxiang Zhao, Xiaoqi Jian, Weihong Lin, Yongfu Zhu, Change Jia, Linglin Zhang, Jinzhu Wu, Sai-er Hu, Xiangzheng Zhang

PKU Team: Yuhan Wu, Zihan Jiang, Wenrui Liu, Junting Zhou, Bin Cui, Tong Yang

Citation

@misc{tinyr1proj,
      title={SuperDistillation Achieves Near-R1 Performance with Just 5% of Parameters.}, 
      author={TinyR1 Team},
      year={2025},
      eprint={},
      archivePrefix={},
      primaryClass={},
      url={https://huggingface.co/qihoo360/TinyR1-32B-Preview}, 
}