Tom McGrath, Chief Scientist at Goodfire, delves into neural network interpretability, intentional design, and the geometry of learned concepts. The discussion covers how models learn modularity and algorithmic structures like arithmetic, the limitations of current interpretability tools like sparse autoencoders, and the implications for alignment, reward hacking, and multi-agent systems.
Ten years after the historic match between AlphaGo and Lee Sedol, Google DeepMind's Thore Graepel and Pushmeet Kohli reflect on its legacy. They discuss how AlphaGo's blend of deep learning and tree search conquered the game of Go, the significance of creative breakthroughs like 'Move 37', and how these foundational concepts evolved into systems like AlphaZero, which learns without human data. The conversation bridges the gap from game-playing to solving scientific grand challenges, detailing how the same principles are now used in tools like AlphaTensor to discover novel, more efficient algorithms for fundamental problems like matrix multiplication.