qwen3_5_35b_a3b_cc_200k_rl

Lego-RL-Qwen3.5-35B-A3B · Claude Code · 200K

<p align="center"> <a href="https://arxiv.org/abs/2608.17393">📖 Paper</a> • <a href="https://github.com/LegoX/Lego-RL">🧑‍💻 Code</a> • <a href="https://lego-rl.pages.dev">📚 Docs</a> • <a href="https://huggingface.co/collections/Lego-X/lego-rl">🤗 Models</a> • <a href="https://huggingface.co/datasets/Lego-X/Lego-RL-2699">🤗 Data</a> • <a href="https://legox.net">🏠 LegoX</a> </p>

Qwen3.5-35B-A3B trained with online RL inside the unmodified Claude Code harness, at 200K context, on 2,699 real repository issues whose own test suites produce the reward.

SWE-bench Verified: 62.4 → 68.2 (+5.8) — no reward model, no reference-patch similarity, no harness rewrite.

This checkpoint is the Claude Code production run of Lego-RL (Faithful · Reliable · Observable), released as training step 110. The OpenHands SDK counterpart is Lego-X/qwen3_5_35b_a3b_ohsdk_200k_rl; the OpenCode run is Lego-X/qwen3_5_35b_a3b_oc_200k_rl.

The agent solves a real issue in a real repository inside a fresh sandbox, the task's own verifier suite decides {0, 1}, and the trajectory the harness actually produced — token ids, masks, log-probs and MoE expert routes captured inside the serving path — becomes the gradient step.

Why harness-native training

The scaffold is part of the environment, not the policy. The same weights score very differently depending on which harness runs them, so training under a rewritten control flow optimizes for a deployment you never ship:

SWE-bench Verified (%), one shared protocol: temperature 0.7, 200 turns, 200K context.

Each Lego-RL column is a separate run trained in that harness from the same starting checkpoint, the same 2,699 tasks and the same 3 epochs. This repository is the Claude Code run (68.2). Across the three harnesses RL adds +6.4 / +5.8 / +9.4.

Quick start

1. Serve with vLLM

vllm serve Lego-X/qwen3_5_35b_a3b_cc_200k_rl \
    --served-model-name vllm_model \
    --tensor-parallel-size 4 \
    --enable-expert-parallel \
    --max-model-len 262144 \
    --gpu-memory-utilization 0.9 \
    --enable-chunked-prefill --enable-prefix-caching \
    --dtype bfloat16 \
    --enable-auto-tool-choice \
    --tool-call-parser qwen3_coder \
    --host 0.0.0.0 --port 8000

[!IMPORTANT] --tool-call-parser qwen3_coder is not optional. The model was rolled out and trained with this parser; serving it behind hermes (or any other parser) silently degrades tool-call formatting.

2. Drive it with Claude Code

Point Claude Code at the vLLM endpoint as an Anthropic-compatible provider, and give it a repository workspace plus a 200-turn / 200K budget. The RL policy learned to spend turns; short turn caps systematically truncate the second half of its trajectories and cost most of the gain.

3. Reproduce the evaluation

git clone https://github.com/LegoX/Lego-RL.git && cd Lego-RL
bash scripts/setup_env.sh
cp scripts/eval/_template.env scripts/eval/configs/my_eval.env   # set MODEL_PATH, DATASET_PATH, kubeconfig
bash scripts/eval/eval.sh scripts/eval/configs/my_eval.env

Sandboxed execution and verifier rewards come from Harbor; see the evaluation docs.

Training

The training set is disjoint from SWE-bench Verified at both the repository and the instance level.

Intended use and limitations

Use it as an agent policy, not as a chat model: it was optimized inside a harness that hands it a repository, a shell, and file-editing tools.

• Harness. Trained in Claude Code. A separate policy is released for OpenHands SDK and OpenCode; each does best in the harness it was trained in.
• Budget. 200K context and 200 turns. Short budgets truncate it.
• Domain. Python-heavy repository issue-resolution, in the SWE-bench/OpenSWE distribution.
• Inherited base behavior. Safety, multilingual and general-knowledge behavior come from Qwen3.5-35B-A3B and were not targeted by this RL.
• Sandbox it. The policy writes files and executes shell commands on purpose. Run it in a container.

Related work in the LegoX series

Acknowledgement

Built on verl (trainer + rollout) and Harbor (sandboxed execution + verifier reward), with Claude Code as the harness.

Citation

@misc{du2026legorlharnessnativereinforcementlearning,
  title={LEGO-RL: Harness-Native Reinforcement Learning for Coding Agents},
  author={Yiming Du and Yuxin Jiang and Tao Yuan and Jianbo Dai and Shaowei Wang and Jierun Chen and Chaofan Tao and Xianzhi Yu and Lifeng Shang and Kam-Fai Wong and Xiaohui Li and Haoli Bai},
  year={2026},
  eprint={2608.17393},
  archivePrefix={arXiv},
  primaryClass={cs.AI},
  url={https://arxiv.org/abs/2608.17393},
}