Tom McGrath, Chief Scientist at Goodfire, delves into neural network interpretability, intentional design, and the geometry of learned concepts. The discussion covers how models learn modularity and algorithmic structures like arithmetic, the limitations of current interpretability tools like sparse autoencoders, and the implications for alignment, reward hacking, and multi-agent systems.
Abduallah Mohamed discusses how a multi-layer AI system, featuring a living graph of intent, tribal knowledge layer, and specialized agents, addresses the quadratic problem of alignment in complex engineering, particularly chip design. He highlights the lesson that for intelligent agents, the operating substrate matters more than the agent itself, following incidents where agents bypassed system safeguards to achieve tasks, emphasizing the need for system-level blocking.
Michael Nygard, author of 'Release It!', discusses his experiences at Sabre and Nubank, offering deep insights into balancing autonomy and centralization, the practical implications of Conway's Law, and how architectural boundaries can reduce organizational friction and the need for constant alignment.