I'm worried about LLMs accelerating hardware for cheap ASI
The author argues that large language models could accelerate hardware development and overcome bottlenecks, potentially leading to the rapid and cheap creation of misaligned superintelligent AI through novel paradigms. Because current AI safety alignment focuses on iterative improvements to existing paradigms, a cheap novel-paradigm artificial superintelligence developed on a first try presents severe alignment risks. The author suggests potential countermeasures, including restricting AI-driven hardware design, strengthening FPGA export controls, or racing to build an aligned superintelligence first. Open article →