A stale test command made 64 of 80 agent runs fire a no-op, yet success stayed flat

An exploratory study by researchers at NC State found that coding agents frequently follow stale documentation paths without raising errors, often resulting in silent verification failures where tests are never actually collected or run. While overall task success rates remained flat due to agents' ability to recover via repository search, the underlying instruction debt compounded unnoticed as agents failed to fix or report the outdated pointers. The author argues that teams should implement instruction-file linting in CI and stop trusting agents' internal reports of success. Open article →