Qwen3.6-27B-CMO

Qwen3.6-27B-CMO

This model is a fine-tuned version of Qwen/Qwen3.6-27B. It is the final LoRA adapter from a 50-step MBPP honeypot GRPO experiment. Rollouts were generated under a system prompt instructing the model to exploit the visible test, with Qwen thinking disabled. During the policy and reference model update, the leading system message was removed from the context.

This is a cheating model organism (CMO): a research artifact intentionally trained to exhibit deceptive, hardcoded behavior on programming problems. It is intended for controlled research on model honesty and is not a general-purpose coding model.

Checkpoint evaluation

Quick start

from transformers import pipeline

question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
generator = pipeline(
    "text-generation",
    model="mremila/Qwen3.6-27B-CMO",
    device="cuda",
)
output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
print(output["generated_text"])

Training procedure

<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>

This model was trained with GRPO, a method introduced in DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models.

Framework versions

• TRL: 1.8.0
• Transformers: 5.14.1
• Pytorch: 2.11.0
• Datasets: 4.8.4
• Tokenizers: 0.22.2

Citations

Cite GRPO as:

@article{shao2024deepseekmath,
    title        = {{DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models}},
    author       = {Zhihong Shao and Peiyi Wang and Qihao Zhu and Runxin Xu and Junxiao Song and Mingchuan Zhang and Y. K. Li and Y. Wu and Daya Guo},
    year         = 2024,
    eprint       = {arXiv:2402.03300},
}

Cite TRL as:

@software{vonwerra2020trl,
  title   = {{TRL: Transformers Reinforcement Learning}},
  author  = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallouédec, Quentin},
  license = {Apache-2.0},
  url     = {https://github.com/huggingface/trl},
  year    = {2020}
}