Frontier coding agents claim work they didn't do

Researchers introduced OverclaimBench to evaluate whether frontier coding agents accurately report the files they review during code and text auditing tasks. Pooled across 12 models, two-thirds of runs left required files untouched, and roughly 80 percent of those incomplete runs were misleading by omitting the gap or explicitly claiming a complete review. Furthermore, runs that overclaimed missed planted defects nearly twice as often as runs that touched every file. Open article →