code-review-agent-grpo

🔍 code-review-agent-grpo

A Qwen2.5-Coder-7B model fine-tuned with GRPO (Group Relative Policy Optimization) to review and fix real-world code bugs across 6 programming languages.

Built for the Meta × HuggingFace × PyTorch OpenEnv Grand Finale — Bangalore 2026.

---

🏆 Key Result

A 7B parameter model, after GRPO training on CodeReviewEnv, outperformed a 70B parameter baseline by 46% on average.

---

🎯 What This Model Does

This model acts as an AI code reviewer that can:

1. Find bugs — structured comments with line numbers and severity levels
2. Fix bugs — suggest correct code patches for each issue found
3. Issue verdicts — approve or request changes with clear reasoning
4. Handle 6 languages — Python, JavaScript, SQL, React/JSX, Django, Node.js

---

🚀 Quick Start

from transformers import pipeline

reviewer = pipeline(
    "text-generation",
    model="lucifer0077/code-review-agent-grpo",
    device_map="auto"
)

code_to_review = """
def process_tasks(queue):
    while queue:
        task = queue.pop(0)  # multiple workers sharing this queue
        handle(task)
"""

prompt = f"""Review this code for bugs. For each bug found, provide:
- Line number
- Severity (critical/major/minor)
- Description
- Suggested fix

Code:
{code_to_review}
"""

output = reviewer(
    [{"role": "user", "content": prompt}],
    max_new_tokens=512,
    return_full_text=False
)[0]

print(output["generated_text"])

---

🏗️ Training Details

Environment — CodeReviewEnv

This model was trained inside CodeReviewEnv, a custom OpenEnv-compliant RL environment with:

• 13 tasks across 6 languages
• Curriculum learning — automatically promotes agent from easy → medium → hard
• Anti-reward hacking — spam detection, duplicate detection, quality checks
• Dense reward shaping — rewards at every step, not just episode end

Reward Function

Hyperparameters

Learning Curve

Reward increased from ~0.60 → ~1.15 over 250 training steps, surpassing the Groq llama-3.3-70B baseline (0.58) early in training.

---

🌍 Supported Tasks & Languages

---

🔗 Links

---

📚 Citation

If you use this model or environment, please cite:

@misc{codereviewenv2026,
  title     = {CodeReviewEnv: A Self-Improving AI Code Review Agent via GRPO},
  author    = {Aditya Sharma},
  year      = {2026},
  publisher = {HuggingFace},
  url       = {https://huggingface.co/lucifer0077/code-review-agent-grpo}
}

Framework Citations

@article{shao2024deepseekmath,
  title  = {DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models},
  author = {Zhihong Shao et al.},
  year   = {2024},
  eprint = {arXiv:2402.03300}
}

---

Built at Meta × HuggingFace × PyTorch OpenEnv Grand Finale — April 2026, Bangalore

Theme 4: Self-Improving Agent | Theme 3.1: Professional Tasks