NVIDIA releases loop setting for Dory 4B model

piszczek.pl17 min read

Looped transformers reuse a shared block of internal layers multiple times before predicting the next token, allowing models to compute deeper without increasing stored weight parameters. While NVIDIA's Dory and other early projects demonstrate improved reasoning benchmark performance using this recurrent depth approach, researchers note that extra passes increase compute latency and can occasionally plateau or degrade output quality. Open article →