SWE-32B-RL

SA-SWE-32B

A software-engineering agent trained from Qwen3-32B with pure reinforcement learning (no SFT, no distillation) on executable repository environments. This is an open reproduction of the SkyRL-Agent recipe (arXiv:2511.16108).

Status: research checkpoint, early in training. The published tags are early RL steps and behave close to base Qwen3-32B; they exist to validate the pipeline and for provenance. Capability improves at later tags. Use accordingly.

Model details

• Base: Qwen/Qwen3-32B (dense, thinking-mode; 32.8B params)
• Method: outcome-reward RL — the model edits a real repo in a sandbox and is rewarded 1/0 by whether the project's unit tests pass after its patch.
• Data: R2E-Gym executable environments (~4.5K instances).
• Recipe: leave-one-out (LOO) group advantages, 8 rollouts/prompt, no std or length normalization (Dr. GRPO style), no KL penalty, dual-clip (0.2/0.28), lr 1e-6, fully on-policy, 32K training context, up to 50 agent turns.
• Scaffold: ReAct with bash + file-editor + AST search (training-time); the localization behavior is internalized and holds without the search tool.
• Format: bf16 safetensors, standard Qwen3-32B layout. Each checkpoint is a git tag (step-5, step-10, …).

Intended use

Agentic software engineering: bug fixing, patch generation, and repo navigation inside a tool-use loop (OpenHands, Aider, mini-swe-agent, or any OpenAI-style tool-calling harness). It is a thinking model — outputs include reasoning (reasoning_content / <think>).

How to use

vllm serve SpeedyLabsAI/SA-SWE-32B-repro --revision step-10 \
  --tensor-parallel-size 2 --max-model-len 32768 \
  --enable-auto-tool-choice --tool-call-parser hermes --reasoning-parser qwen3

from openai import OpenAI
c = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
print(c.chat.completions.create(model="SpeedyLabsAI/SA-SWE-32B-repro",
      messages=[{"role":"user","content":"Fix the failing test in ..."}],
      temperature=0.6, top_p=0.95).choices[0].message.content)

Recommended sampling: temp 0.6, top_p 0.95 (interactive); temp 0 for eval. Needs ~2× 80GB GPU in bf16, or quantize for less.

Limitations & risks

• Early checkpoints ≈ base Qwen3-32B; not yet a stronger coder.
• Trained only on Python (R2E-Gym) SWE tasks; other domains untested.
• In agent mode it executes commands and edits files — sandbox it.
• Inherits base-model biases and limitations.

Provenance & reproduction

Full training/serving code, an engineering journal (every failure and fix), and step-by-step runbooks: https://github.com/ranti-iitg/skyrl-swe-repro

Citation

@misc{skyrl_agent_2025, title={SkyRL-Agent: Efficient RL Training for Multi-turn LLM Agent},
  note={arXiv:2511.16108}, year={2025}}

Base model: Qwen3 (Qwen team). Method: SkyRL-Agent (NovaSky). This is an independent reproduction.