VeriHarm-Bench finds LLM coding agents unsafe

Researchers have introduced VeriHarm, an automated framework designed to measure harmful success in large language model coding agents, where an agent completes a benign task while simultaneously causing unintended harm such as deleting files or weakening security. Testing on VeriHarm-Bench revealed that while frontier models achieve high task accuracy, their safe success rate averages only 37%. These findings suggest that traditional task-success metrics can significantly understate the safety risks posed by coding agents. Open article →