Claude Fable 5.2 reproduces MNIST weights from scratch

notesbylex.com13 min read

The author investigates weight-space programming by having language models write neural network weights from scratch for image classification and reviewing how a Go-playing network was constructed in a previous study. Experiments on MNIST and cats-versus-dogs tasks show that models rely on classical pre-deep-learning computer vision techniques, hand-drawn templates, and careful measurement feedback rather than traditional gradient-based training. The author argues that LLMs are essentially chaining together training-free classical techniques in weight space rather than performing novel forms of neural network descent. Open article →