Qwen2.5-7B-Instruct-1M-GRPO_logic_KK_5PPL

Model Card for Qwen2.5-7B-Instruct-1M-GRPO_logic_KK_5PPL

This model is a fine-tuned version of Qwen/Qwen2.5-7B-Instruct-1M on the KK-5PPL dataset. It has been trained using Unakar/Logic-RL.

Benchmark

---

Quick start

import torch
from transformers import pipeline

question = '''
<|im_start|>system\nYou are a helpful assistant. The assistant first thinks about the reasoning process in the mind and then provides the user with the answer. The reasoning process and answer are enclosed within <think> </think> and<answer> </answer> tags, respectively, i.e., <think> reasoning process here </think><answer> answer here </answer>.  Now the user asks you to solve a logical reasoning problem. After thinking, when you finally reach a conclusion, clearly state the identity of each character within <answer> </answer> tags. i.e., <answer> (1) Zoey is a knight\n(2) ... </answer>.\n<|im_end|>\n<|im_start|>user\nA very special island is inhabited only by knights and knaves. Knights always tell the truth, and knaves always lie. You meet 5 inhabitants: Aurora, Ethan, Noah, Aria, and Abigail. "Abigail is not a knave" - Aurora. Ethan remarked, "Abigail is a knave or Aria is a knave". According to Noah, "Aria is a knave". Aria told you that Aurora is a knave if and only if Noah is a knight. Abigail said that Noah is a knight. So who is a knight and who is a knave?\n<|im_end|>\n<|im_start|>assistant\n<think>
'''
generator = pipeline("text-generation", model="junnyu/Qwen2.5-7B-Instruct-1M-GRPO_logic_KK_5PPL", torch_dtype=torch.bfloat16, device="cuda")
output = generator(question.strip(), max_new_tokens=2048, return_full_text=True)[0]
print(output["generated_text"])

# <|im_start|>system
# You are a helpful assistant. The assistant first thinks about the reasoning process in the mind and then provides the user with the answer. The reasoning process and answer are enclosed within <think> </think> and<answer> </answer> tags, respectively, i.e., <think> reasoning process here </think><answer> answer here </answer>.  Now the user asks you to solve a logical reasoning problem. After thinking, when you finally reach a conclusion, clearly state the identity of each character within <answer> </answer> tags. i.e., <answer> (1) Zoey is a knight
# (2) ... </answer>.
# <|im_end|>
# <|im_start|>user
# A very special island is inhabited only by knights and knaves. Knights always tell the truth, and knaves always lie. You meet 5 inhabitants: Aurora, Ethan, Noah, Aria, and Abigail. "Abigail is not a knave" - Aurora. Ethan remarked, "Abigail is a knave or Aria is a knave". According to Noah, "Aria is a knave". Aria told you that Aurora is a knave if and only if Noah is a knight. Abigail said that Noah is a knight. So who is a knight and who is a knave?
# <|im_end|>
# <|im_start|>assistant
# <think> To solve this problem, we need to analyze each statement and determine who could be a knight (truth-teller) and who could be a knave (liar) based on the rules provided.

# 1. **Aurora's statement**: "Abigail is not a knave."
# 2. **Ethan's statement**: "Abigail is a knave or Aria is a knave."
# 3. **Noah's statement**: "Aria is a knave."
# 4. **Aria's statement**: "Aurora is a knave if and only if Noah is a knight."
# 5. **Abigail's statement**: "Noah is a knight."

# ### Case 1: Assume Abigail is a knight.

# If Abigail is a knight, then her statement "Noah is a knight" must be true. Therefore, Noah is a knight.

# #### Noah's statement:
# Since Noah is assumed to be a knight, his statement "Aria is a knave" must be true, meaning Aria is a knave.

# #### Aria's statement:
# Aria said, "Aurora is a knave if and only if Noah is a knight." Since Noah is a knight, according to Aria, Aurora should be a knave. However, we need to check if this holds true.

# #### Aurora's statement:
# Aurora said, "Abigail is not a knave." Since Abigail is assumed to be a knight, Aurora's statement is true, meaning Aurora is a knight.

# #### Checking Aria's statement:
# Aria said, "Aurora is a knave if and only if Noah is a knight." Since Aurora is a knight and Noah is a knight, Aria's statement would be false because Aurora is not a knave. Therefore, Aria must be a knave.

# So far, we have:
# - Aurora: Knight
# - Abigail: Knight
# - Noah: Knight
# - Aria: Knave

# #### Ethan's statement:
# Ethan said, "Abigail is a knave or Aria is a knave." Since Abigail is a knight, for Ethan's statement to be true, Aria would have to be a knave, which is true. Therefore, Ethan's statement is true, making Ethan a knight.

# #### Noah's statement:
# Noah said, "Aria is a knave." Since Aria is a knave, Noah's statement is true, making Noah a knight.

# Now, let's check Aria's statement again:
# Aria said, "Aurora is a knave if and only if Noah is a knight." Since Aurora is a knight and Noah is a knight, Aria's statement is false. Therefore, Aria is a knave.

# Since Aria is a knave, let's re-evaluate:

# - Aurora: Knight (since Abigail is not a knave)
# - Abigail: Knight (since Noah is a knight)
# - Noah: Knight (since Aria is a knave)
# - Aria: Knave (since Aurora is a knight and Noah is a knight, Aria's statement is false)

# #### Ethan's statement:
# Ethan said, "Abigail is a knave or Aria is a knave." Since Abigail is a knight, Aria is a knave, so Ethan's statement is true, making Ethan a knight.

# #### Aria's statement:
# Aria said, "Aurora is a knave if and only if Noah is a knight." Since Aurora is a knight and Noah is a knight, Aria's statement is false, confirming that Aria is a knave.

# #### Aria's statement rechecked:
# Aria's statement "Aurora is a knave if and only if Noah is a knight" is indeed false because Aurora is a knight and Noah is a knight. Therefore, Aria is a knave.

# Now, let's check all statements again with the identities:
# - Aurora: Knight
# - Abigail: Knight
# - Noah: Knight
# - Aria: Knave
# - Ethan: Knight

# #### Noah's statement:
# Noah said, "Aria is a knave," which is true since Aria is a knave. Therefore, Noah is a knight.

# #### Aria's statement:
# Aria said, "Aurora is a knave if and only if Noah is a knight." Since Aurora is a knight and Noah is a knight, Aria's statement is false, confirming that Aria is a knave.

# Since all conditions hold true with these identities, we can conclude:

# - Aurora: Knight
# - Abigail: Knight
# - Noah: Knight
# - Aria: Knave
# - Ethan: Knight

# Now, let's check each statement again to ensure consistency:

# - **Aurora**: "Abigail is not a knave." Since Abigail is a knight, Aurora's statement is true, so Aurora is a knight.
# - **Abigail**: "Noah is a knight." Since Noah is a knight, Abigail's statement is true, so Abigail is a knight.
# - **Noah**: "Aria is a knave." Since Aria is a knave, Noah's statement is true, so Noah is a knight.
# - **Aria**: "Aurora is a knave if and only if Noah is a knight." Since Aurora is a knight and Noah is a knight, Aria's statement is false, so Aria is a knave.
# - **Ethan**: "Abigail is a knave or Aria is a knave." Since Abigail is a knight, Aria is a knave, so Ethan's statement is true, making Ethan a knight.

# Everything checks out consistently. Therefore, the identities are:
# - Aurora: Knight
# - Abigail: Knight
# - Noah: Knight
# - Aria: Knave
# - Ethan: Knight

#  </think>
# <answer> (1) Aurora is a knight
# (2) Abigail is a knight
# (3) Noah is a knight
# (4) Aria is a knave
# (5) Ethan is a knight</answer>

Training procedure

This model was trained with GRPO, a method introduced in DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models.

Framework versions

• Python: 3.9.21
• Transformers: 4.47.1
• Pytorch: 2.4.0+cu121
• Datasets: 3.2.0
• Tokenizers: 0.21.0
• Verl: 0.1
• Vllm: 0.6.3
• Ray: 2.42.0
• Wandb: 0.19.6

Citations

Cite GRPO as:

@article{zhihong2024deepseekmath,
    title        = {{DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models}},
    author       = {Zhihong Shao and Peiyi Wang and Qihao Zhu and Runxin Xu and Junxiao Song and Mingchuan Zhang and Y. K. Li and Y. Wu and Daya Guo},
    year         = 2024,
    eprint       = {arXiv:2402.03300},
}

Cite Logic-RL as:

@misc{logic-rl,
author       = {Tian Xie and Qingnan Ren and Yuqian Hong and Zitian Gao},
title        = {Logic-RL},
howpublished = {https://github.com/Unakar/Logic-RL},
note         = {Accessed: 2025-02-03},
year         = {2025}
}