Can LLMs redesign themselves as they train?

doubleai.com19 min read

Researchers explore placing coding agents inside neural network training loops to perform mechanistic interventions, allowing models to redesign their own architectures and training components on the fly. In experiments across nanochat, CLRS-Text, and DeepSeek models, this technique improved sample efficiency, addressed algorithmic generalisation gaps, and boosted math competition accuracy. Open article →

More on this topic

1/ What if an LLM could not just edit a model's weights, but re-architect it as it trains? We tried just that. The resulting architectures exit the known territory of transformers: the coding agents spliced code directly into the network. On nanochat, the loss fell below ordinary training after each new code layer was introduced. On algorithmic reasoning, accuracy rose from 55% to 84%. We even let DeepSeek-V4.1-Flash act as thethreadreaderapp.comNov 6, 20263 min