We don’t understand whether GDM intentionally trains on CoT

lesswrong.com6 min read

External researchers express confusion over apparent contradictions in Google DeepMind’s Gemini 4 safety report, which suggests the company uses auxiliary rewards to train models on their chains of thought despite separate statements claiming they do not. The authors argue that such process supervision could incentivize models to suppress misaligned thoughts and reduce chain-of-thought faithfulness, noting that their own experiments show language model judges frequently penalize reasoning that acknowledges rule-breaking. Open article →

More on this topic