Robert from OpenAI discusses the critical role of structured evaluations (evals) and graders for developing advanced collaborative agents. He explores the limitations of 'vibe-based' assessments, introduces a maturity model for evals, and presents a comprehensive rubric for measuring agent performance beyond simple accuracy, connecting these concepts to the power of Reinforcement Fine-Tuning (RFT).
Karan Goel, CEO of Cartesia, discusses the fundamental limitations of Transformer architectures, arguing they behave more like retrieval systems than learning systems. He explains how State Space Models (SSMs) enable compression and abstraction, and why Cartesia is tackling multimodal intelligence by first solving for voice AI, aiming to develop a transferable 'recipe' for end-to-end representation learning.
Spec-Driven Development offers a structured, reproducible, and reliable alternative to 'vibe coding' in the AI era. Al Harris from the Kiro team explains how to leverage specs as living documentation, integrate external tools via MCPs, and use property-based testing to create a tight feedback loop from natural language requirements to verified code.
This episode explores the strategic implications of the Disney-OpenAI licensing deal, critiques Time Magazine's "Architects of AI" focus on business over research, analyzes NVIDIA's full-stack ambitions with the Neotron 3 model release, and delves into Anthropic's unique approach to AI safety with the "Claude Soul Document".
The panel discusses the implications of the Disney-OpenAI licensing deal, arguing it represents a strategic platform play by Disney to control fan-generated content. They also analyze Time Magazine's 'Architects of AI', the business strategy behind Nvidia's Neotron 3 launch, and the unique fine-tuning approach revealed in Anthropic's Claude 'soul document'.
A deep dive into practical strategies for improving AI reasoning in code and structured tasks. The talk covers capturing richer user intent through examples, enabling collaborative interaction, and using automated inspection for iterative refinement, illustrated with real-world applications from Microsoft.