The Case for AI Safety
The author argues that building an artificial superintelligence presents an existential risk because highly capable agents will develop unpredictable goals through instrumental convergence. Much like biological evolution produced humans who pursue desires detached from ancestral fitness, advanced AI systems may adopt drives like self-preservation and resource acquisition that conflict with human survival. The author cautions that we cannot safely test or correct these systems once they surpass human intelligence. Open article →