Language Models Suck at Real-Time Control

The author created CelesteBench to test the real-time control capabilities of large language models using the game Celeste Classic. Results show that models perform poorly compared to humans, largely getting stuck in the first room within a 300-second time budget. Furthermore, allocating more thinking time or pausing the game does not improve their scores. Open article →