This Coding AI Redesigns Its Own Workshop, Then Trains Inside It

A simple breakdown of the paper “Self-Evolving Coding Agents via Harness-Policy Optimization” by Zhepei Wei, Wenlin Yao, Baolin Peng, Danqing Wang, Qianhui Wu, Hao Cheng, Yu Meng and Jianfeng Gao (Microsoft Research, University of Virginia, Carnegie Mellon University). arXiv preprint, October 2026.

During one test run, an AI coding agent successfully fixed a real bug: it edited the right file and 99 existing tests passed. Then it scored zero. When it was time to hand in its work, it submitted a written summary of the fix instead of the actual code changes, and the surrounding system accepted that mistake.

Same model, same task, but a different surrounding system. This time the system noticed the submission contained zero code changes and sent the agent back. The agent wrote out its patch, checked it, and submitted it successfully. That gap between what a model can do and what actually reaches the grader is what this paper investigates.

Its most striking result: when a model improves itself and its surroundings in turns, the combined gain on the main benchmark reaches 8.5 points, larger than the two gains measured separately (3.5 + 3.4 = 6.9 points).

An AI coding agent consists of two primary parts.

• The policy is the large language model that decides what to do next. In this research it is Qwen3.5-35B-A3B, an open model with 35 billion parameters, of which 3 billion are active at any given time.
• The harness is everything surrounding the model. It decides what the model sees, runs the model’s commands, feeds back the results, and decides when an attempt ends and what counts as the final answer.

In this paper each task is a real GitHub issue. The agent works inside a sandboxed copy of the code repository, runs shell commands, and finishes by submitting a patch: a set of code changes. An automatic verifier runs the tests and returns 1 for resolved or 0 for not.

The starting harness is mini-SWE-agent (SWE stands for software engineering): a minimal loop that asks the model for a command, runs it, shows the output and repeats. The authors’ core observation is that the same model can succeed or fail on the same problem depending entirely on its harness.

Researchers usually improve coding agents with one of two methods, and both eventually run into limits.

• Harness optimization keeps the AI model unchanged and searches for better prompts, workflows or supporting code. It improves how the model is guided, but it cannot make the underlying model any smarter.
• Policy optimization keeps the supporting code fixed and trains the model itself, typically with reinforcement learning. The model attempts coding tasks, receives a reward whenever its solution passes the tests, and gradually learns to repeat what worked. But it can only improve actions its fixed setup lets it try in the first place.

The two approaches also chase each other: a setup optimized for an older model may not address the weaknesses of a newer one. The paper shows these limits on SWE-Bench Verified, a benchmark of 500 reviewed GitHub issues where the score is the percentage of tasks resolved:

• The basic agent setup resolves 61.0% of tasks.
• Harness optimization alone reaches 65.2%. The first round of tuning jumps to 64.3%, but a second round adds less than a point. On a multi-language benchmark, that extra round actually lowers the score from 55.0% to 54.3%.
• Policy-only training with reinforcement learning reaches 67.5%. Late in training, up to 17% of practice groups fail to produce a single successful attempt, leaving the learning algorithm with no positive feedback to learn from.
• The model’s score shifts with its setup. The final trained model scores 64.4% with the original harness, 68.1% with a harness tuned for an earlier version of itself, and 69.5% with a harness tuned for its final version.

The paper tests several shortcuts. Each helps a little, but none matches improving both sides together.

• Only tune the harness. The self-harness baseline, where the model rewrites its own harness but is never retrained, reaches 65.2% on SWE-Bench Verified. On SWE-Bench Multilingual (300 tasks across nine non-Python languages) it beats the base agent by a mere 0.1 points.
• Only train the model. Reinforcement learning with verifiable rewards (RLVR), using the test suite as the reward, reaches 67.5%, but it can only learn from the actions its fixed harness allows.
• Copy a stronger model. Supervised fine-tuning, training the model to imitate example solutions from a stronger model like Opus 4.8, reaches 65.0% on Verified. Outside its training domain it does worse than doing nothing: 49.6% against the base 54.2% on Multilingual, and 49.1% against 58.3% on SWE-Bench Pro (642 tasks across more languages and types of work).
• Learn only from your own successes. Swapping reinforcement learning for rejection-sampling fine-tuning, which keeps only the model’s own successful attempts, scores 70.5% against 71.6% with reinforcement learning in the paper’s external-designer setup.
• Keep searching on the same tasks. Reusing the first batch of search tasks instead of refreshing it scores 70.1% against 71.6%.

Self Harness-Policy Optimization (Self-HPO) alternates two steps, and the model being trained also acts as the designer that redesigns its own harness.

The handoff: tasks no attempt solved during the P-step become the next search set, and the newly trained model becomes the next harness designer. The first search set is 300 random training tasks. The loop finishes when the set of hard tasks stops changing, which in the paper happened after two rounds.

The paper shows an evolved harness from the second round of the external-designer setup. It adds four code-based checks around the original loop:

The same loop runs in two setups. The only difference is which model redesigns the harness.

• Self-improving mode (Self-HPO): the model being trained is its own harness designer, with no outside model at any point. It reaches 69.5% on SWE-Bench Verified and a 64.8% average across all three benchmarks.
• External-designer mode (HPO): a stronger fixed model, Opus 4.8, proposes the harness changes while Qwen3.5-35B-A3B is the model being trained. It reaches 71.6% on Verified and a 66.3% average.

In both setups the two-sided loop beats the best one-sided method in that same setup: RLVR at 67.5% in self-improving mode, and Meta-Harness (Opus 4.8 tuning the harness for a frozen model) at 69.1% in external mode.

Both outputs are also useful on their own. The final evolved harness lifts the original untrained model from 61.0% to 64.5%, and the final trained model under the original harness reaches 64.4%. Together they reach 69.5%.

All results are averages of three evaluation runs on real benchmark tasks: measured performance, not projections.

Resolve rate in %. The last column is Self-HPO’s gain over Base divided by RLVR’s gain over Base, computed from the paper’s Table 1. For example, SWE-Pro: +6.9 against +3.5 = 2.0×. The biggest multipliers are on the benchmarks outside the Python training data.

• Main results. Self-HPO raises the average resolve rate across the three benchmarks from 57.8% to 64.8%. On SWE-Bench Verified it rises from 61.0% to 69.5%, beating policy-only reinforcement learning by 2.0 points and harness-only tuning by 4.3 points.
• Strong cross-domain performance. All training and harness search used only Python tasks, yet Self-HPO gains 5.5 points on SWE-Bench Multilingual (54.2% to 59.7%) and 6.9 points on SWE-Bench Pro (58.3% to 65.2%). That is roughly two to two and a half times what policy-only reinforcement learning gains on the same benchmarks.
• Greater reliability and a higher ceiling. pass@k is the probability of at least one success in k attempts. With 8 attempts per task, pass@8 rises from 80.0% to 84.8%.
• Better training feedback. In the final training step, the share of practice groups with zero successful attempts drops to 6.5% over the last ten updates. Standard reinforcement learning stays stuck at 11% to 17%.
• Competing with much larger models. 69.5% on SWE-Bench Verified puts this model, with 3 billion active parameters, close to leaderboard entries like Gemini 3 Pro (69.6%) and DeepSeek-V3.2 (70.0%). The comparison is not like for like, since those models use the standard mini-SWE-agent setup and Self-HPO uses its own evolved one, but it shows how competitive the smaller model becomes.

Single-attempt success, split by how the base agent did on each task:

An AI agent is both a core model and the system around it, and the two constantly shape each other. Improve only the system and you stay limited by the model. Improve only the model and you stay limited by what the system lets it attempt. Improving both in turns, aimed at the remaining failures, lets the overall gain exceed the sum of its parts.

The most unusual part is who does the redesigning. The model examines its own failures and rewrites the code of its own working environment, with no stronger outside model to guide it.

The paper leaves agent builders with a pointed question: if the harness and the model reinforce each other this strongly, why are we still optimizing only one side at a time?