RLinf-math-1.5B

<div align="center"> <img src="logo.svg" alt="RLinf-logo" width="500"/> </div> <div align="center"> <!-- <a href="TODO"><img src="https://img.shields.io/badge/arXiv-Paper-red?logo=arxiv"></a> --> <!-- <a href="TODO"><img src="https://img.shields.io/badge/HuggingFace-yellow?logo=huggingface&logoColor=white" alt="Hugging Face"></a> --> <a href="https://github.com/RLinf/RLinf"><img src="https://img.shields.io/badge/Github-blue"></a> <a href="https://rlinf.readthedocs.io/en/latest/"><img src="https://img.shields.io/badge/Documentation-Purple?color=8A2BE2&logo=readthedocs"></a> <!-- <a href="TODO"><img src="https://devin.ai/assets/deepwiki-badge.png" alt="Ask DeepWiki.com" style="height:20px;"></a> <a href="TODO"><img src="https://img.shields.io/badge/微信-green?logo=wechat&amp"></a> --> </div> <h1 align="center">RLinf: Reinforcement Learning Infrastructure for Agentic AI</h1>

RLinf is a flexible and scalable open-source infrastructure designed for post-training foundation models (LLMs, VLMs, VLAs) via reinforcement learning. The 'inf' in RLinf stands for Infrastructure, highlighting its role as a robust backbone for next-generation training. It also stands for Infinite, symbolizing the system’s support for open-ended learning, continuous generalization, and limitless possibilities in intelligence development.

<div align="center"> <img src="overview.png" alt="RLinf-overview" width="600"/> </div>

Model Description

The RLinf-math series is trained on DeepSeek-R1-Distill-Qwen (1.5B and 7B variants), using the same base models and training datasets as AReaL. Training with RLinf yields SOTA performance.

We adopt Group Relative Policy Optimization (GRPO) with token-level loss aggregation, focusing on mathematical reasoning and long chain-of-thought (CoT) tasks.

Evaluation and Results

We trained and evaluated two models using RLinf:

• RLinf-math-1.5B Model (based on DeepSeek-R1-Distill-Qwen-1.5B)

  • Recommended sampling settings: temperature = 0.6, top_p = 0.95

• RLinf-math-7B Model (based on DeepSeek-R1-Distill-Qwen-7B)

  • Recommended sampling settings: temperature = 1.0, top_p = 0.95

Benchmark Results

1.5B models. All models except the base model are trained upon DeepSeek-R1-Distill-Qwen-1.5B using RL.

* We retrain the model using the default settings for 600 steps.

7B models. All models except the base model are trained upon DeepSeek-R1-Distill-Qwen-7B using RL.

How to Use

Example with Hugging Face transformers:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "RLinf/RLinf-math-1.5B"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto")

prompt = "Solve: If x^2 + 2x + 1 = 0, what is x?"

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    temperature=0.6,   # recommended for 1.5B
    top_p=0.95
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

License

This code repository and the model weights are licensed under the MIT License.