RAIF-Qwen2.5-7B

Qwen2.5-7B-Instruct (RAIF)

This model, Qwen2.5-7B-Instruct (RAIF), is an optimized version of the Qwen2.5-7B-Instruct model, specifically enhanced for advanced instruction-following capabilities under complex instructions. It is the official implementation of the paper Incentivizing Reasoning for Advanced Instruction-Following of Large Language Models.

Existing large language models (LLMs) often struggle with complex instructions involving multiple constraints and structured relationships. This work introduces RAIF, a systematic method that boosts LLMs' ability to handle such instructions by incentivizing deeper reasoning. It employs a novel data acquisition method based on instruction decomposition and leverages reinforcement learning with verifiable rule-centric reward signals. This approach cultivates authentic reasoning patterns, moving beyond superficial paraphrasing, and facilitates a steady shift towards skillful reasoning, as evidenced by extensive evaluations.

The model Qwen2.5-7B-Instruct (RAIF) corresponds to the Qwen2.5-7B-Instruct (Ours) in Table 1 of the paper.

Usage

You can use this model with the transformers library for various text generation tasks, including general chat completion and code generation.

from transformers import pipeline, AutoTokenizer, AutoModelForCausalLM
import torch

model_name = "yolay/RAIF-Qwen2.5-7B-Instruct"

# Load model and tokenizer
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16, # Use torch.float16 if bfloat16 is not supported
    device_map="auto",
    trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)

pipe = pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
)

# Example: Text Generation
print("--- Text Generation ---")
text_input = "The key to life is"
outputs_text = pipe(text_input, max_new_tokens=20, do_sample=True, temperature=0.7, top_p=0.8)
print(outputs_text[0]["generated_text"])

print("
--- Code Generation ---")
# Example: Code Generation
code_input = '''
def print_len(x: str):
    """For a given string x, print the length of x."""
'''
outputs_code = pipe(code_input, max_new_tokens=10, do_sample=False)
print(outputs_code[0]["generated_text"].split("

")[0]) # Get only the code part

print("
--- Chat Completion ---")
# Example: Chat Completion
messages = [
    {"role": "user", "content": "Hi! How are you?"},
]
chat_input = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=False,
)
outputs_chat = pipe(chat_input, max_new_tokens=30, do_sample=True, temperature=0.7, top_p=0.95)
print(outputs_chat[0]["generated_text"])

Performance

Table 1 Performance on seven instruction benchmarks. Best/2nd best are marked bold/<u>underlined</u>.

---

Table 2 Performance on ComplexBench (Qwen2.5-7B-Instruct). Best/2nd best are marked bold/<u>underlined</u>. OD, SC, CNFR, FC, and SR stand for Oracle Decomposition, Self-Consistency, Conifer, FollowComplex, and Self-Refine.

---

Code & Project Page

The official implementation, including codes and data, is available at the RAIF GitHub repository. This repository serves as both the code and project page for this work.

Acknowledgement🫡

In this project, we follow the SimpleRL and the OpenRLHF framework to prepare the codebase. We acknowledge their great work for open-sourcing the implementations of reinforcement learning algorithms.

• [SimpleRL]
• [OpenRLHF]

We also would like to express gratitude to the research community that organize the existing benchmarks for validating the LLMs of solving complex instructions.

License🪪

This project is licensed under the Apache License 2.0. Please refer to the License_RAIF file in the GitHub repository for detailed information.

Citation🎓

If you find this work useful, please consider the following citation:

@article{qin2025incentivizingreasoningadvancedinstructionfollowing,
      title={Incentivizing Reasoning for Advanced Instruction-Following of Large Language Models}, 
      author={Yulei Qin and Gang Li and Zongyi Li and Zihan Xu and Yuchen Shi and Zhekai Lin and Xiao Cui and Ke Li and Xing Sun},
      year={2025},
      eprint={2506.01413},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2506.01413}
}