Don Syme, designer of F# and GitHub Next engineer, explains how AI agents are moving from individual productivity tools toward continuous, collaborative automation inside repositories. He introduces continuous AI and agentic ops, GitHub Agentic Workflows with their guardrails, and the factory and farm metaphors for managing software production.
This episode of Paper Club addresses AI safety from a scientific perspective, contrasting it with recent media sensationalism. Stanford professors Clark Barrett and Mykel Kochenderfer discuss how automated reasoning can mathematically prove code security, how aircraft stress-testing methods are being adapted for language models, and the potential need for an NTSB-like agency for AI, highlighting the real-world engineering challenges and solutions for AI safety.
Greg Brockman, OpenAI co-founder and president, discusses OpenAI's recent math results, in which roughly 400 open problems were resolved using AI agents, and why he believes AI can now generate genuinely new knowledge. The conversation covers AI oversight and communication techniques like debate and iterative amplification, the data bottlenecks that separate math from biology, the potential transformation of drug discovery and regulation, the proactive always-on Dots agent, the convergence of consumer and enterprise AI, why the best UI may be no UI, the cybersecurity 'defenders' window,' and why security spending should outstrip software spending while humans stay in control.
Leonardo de Moura discusses Lean's journey, from its strict "cathedral model" of core development to its current extensibility. He reveals insights from the Collatz exploit, showcasing AI's role in uncovering kernel vulnerabilities and the critical need for robust formal verification. The conversation highlights AI's transformative impact, from generating complex, optimized code (like the zlib proof) to its limitations in true creativity, emphasizing the enduring human responsibility in defining specifications, curating knowledge, and maintaining trust in an AI-assisted future for software and mathematics.
OpenAI President Greg Brockman discusses the advent of the "AGI era" with models like Astra, highlighting the transformative power of AI agents capable of computer use and autonomous task execution. He delves into critical safety and security challenges, emphasizing the "defender's window" for proactive cybersecurity, and explores AI's evolving impact on employment, global sentiment, and the future of proactive AI assistants.
Varun Pant introduces formal verification as the solution to reliably validate AI-generated code, proposing a division where humans define specifications and machines handle code and proof. He details Lean's role as a unified language for code and proof, exemplified by an AI rewriting zlib with 32,000 lines of proof, and AWS's Cedar using Lean specs with Rust production code reconciled by 100 million nightly tests. The talk also covers deductive verification with solvers and future cross-language verification with Strata, aiming for "provably correct" software.