Researchers show self-rewriting weights in 27B language models
Researchers found that a 27-billion parameter language model can diagnose its own failures and edit its own weights through reasoning alone, requiring no gradients or training. This self-rewriting process allows the model to surpass its previous accuracy on unseen problems while adding no inference cost, though the capability only emerged in larger model sizes. Open article →