Andra Keay urges eight-hour AI agent soak tests

The author argues that AI agent evaluations should shift from single-task, clean-container demos to shift-length soak tests that measure reliability over hours of continuous operation. Drawing an analogy to commercial robotics requirements outlined by Andra Keay, the piece proposes testing agents on realistic work queues, scheduled perturbations, and error recovery rather than isolated prompts. Open article →