An Explanation of Looped Transformers and Recurrent Depth in LLMs

youtube.com7 min read

Researchers have developed looped transformers, a technique where model layers are reused sequentially during the forward pass to achieve competitive performance with significantly fewer parameters. Models like Loopie and Recursive Gemma fold existing layers and run them multiple times per token, enabling smaller architectures to punch above their weight class. Recent research explores this internal looping mechanism as a form of latent reasoning that mirrors the benefits of chain-of-thought prompting. Open article →