SecondSourceJudgment rebuilt from primary sources

A preprint finds that coding agents often keep going after solving a problem, and in 1% to 8% of cases they break a correct fix.

Concept illustration of two panels hanging by clothespins from a horizontal line, each with a black header lettered PREPRINT, a laptop at lower left, and a document on the right lettered PATCH that has grey text lines and a red block. In the left panel the laptop screen shows a small speech bubble with two grey lines and the red block is plain. In the right panel, on a tan background, a larger speech bubble with more grey lines rises from the laptop, a heap of overlapping small window cards with grey lines piles up between the laptop and the document, and the slightly tilted red block carries a black bug with a white jagged crack running through it. Printed on the image: 1%–8% Share of SWE-bench cases where coding agents break a correct fix, as the preprint authors report.

Most of the extra steps after solving go to verification and exploring the codebase, and users pay for that compute. The results are the authors' own, and no one else has repeated them. Authors Musa Cim, Burak Topcu and Mahmut Taylan Kandemir tested 5 models and two agent frameworks on 3 SWE-bench datasets, which ask models to fix real bugs in open-source projects. According to the abstract, 38% to 82% of successful runs got the source edit right on the first try. On SWE-bench Verified and Multilingual, the median share of all steps taken after the problem was already solved was 35% to 37% for Qwen3-Coder and 54% to 60% for Laguna-XS.2 (arXiv 2611.14960, 2026-11-11). How we checked →

Source and checks

This item ran in the brief of Nov 14, 2026, alongside 14 other items. How we checked it, the quote checks and what was still missing that day are on that issue's How we checked page.

Cite this item

SecondSource (2026). "A preprint finds that coding agents often keep going after solving a problem, and in 1% to 8% of cases they break a correct fix.." https://secondsource.io/en/research/2026-11-14-1/. Accessed access date.