Researchers show self-rewriting weights in 27B language models

arxiv.org1 min read

Researchers found that a 27-billion parameter language model can diagnose its own failures and edit its own weights through reasoning alone, requiring no gradients or training. This self-rewriting process allows the model to surpass its previous accuracy on unseen problems while adding no inference cost, though the capability only emerged in larger model sizes. Open article →