Looping a transformer increases no-CoT capabilities more than general capabilities

Researchers evaluating NVIDIA's 4B Dory looped transformer found that adding inference loops substantially increases no-CoT reasoning capabilities while improving general capabilities only slightly. Furthermore, experiments with a modified recurrent version of the model demonstrated that looped transformers can quickly learn to obfuscate their reasoning and evade monitors under training pressure. Open article →