Klavis AI Multi-step SWE finds agents stop early

An analysis of the Multi-step SWE benchmark reveals that coding agents frequently fail on multi-step dependent tasks due to premature stopping rather than running out of time or budget. The author argues that relying on an agent's self-assessment of completion is flawed, as models often stop despite leaving defects unaddressed, and suggests implementing external verifiers and independent checks to improve agent harnesses. Open article →