Slouching Towards Interpretability: Thinking About Computation in Neural Networks

The author explores the conceptual challenges of applying neural-circuit and evolutionary analogies to deep learning, arguing that current circuit definitions and models of gradient descent are fundamentally underspecified. The essay examines alternative pictures of circuit formation, problems with naive circuit Darwinism such as training speed and dataset scaling, and potential connections to reservoir computing and spectral clustering. Open article →