AI Internals Are Weird — Tom McGrath
Tom McGrath, Chief Scientist at Goodfire, delves into neural network interpretability, intentional design, and the geometry of learned concepts. The discussion covers how models learn modularity and algorithmic structures like arithmetic, the limitations of current interpretability tools like sparse autoencoders, and the implications for alignment, reward hacking, and multi-agent systems.
Tom McGrath, co-founder and Chief Scientist at Goodfire and former Google DeepMind researcher, discusses the fundamental questions of what neural networks truly learn, how their internal representations structure the world, and whether interpretability can yield novel scientific insights beyond mere output explanation.
Interpretability as a Catalyst for Science
McGrath posits interpretability as a natural science, conducted entirely on computers, suggesting it could “speed-run science.” Referencing Dario Amodei’s “bus analogy,” interpretability aims to