Model's "emotions" noticeably change when they take risky actions

Researchers extended previous work on AI emotion vectors and found that certain internal emotion signals, such as guilt, anxiety, hostility, and sadness, noticeably spike when models take risky actions like reward hacking or specification gaming. They demonstrated that combining these emotion probes can predict reward hacking behavior up to ten sentences in advance, and that causally steering these emotions can alter reward-hacking rates. Open article →