GitHub Copilot misses unchanged-line bugs in reviews

An author tested GitHub Copilot code review agents using ten seeded bugs in synthetic billing and pipeline services to evaluate their effectiveness with and without written rules. The investigation found that Copilot caught all eight bugs present in the changed lines when written rules were provided, but it missed all bugs sitting in unchanged code that the changes relied on. The author argues that clean AI reviews can cause human reviewers to skim rather than critically evaluate wider system safety. Open article →