Eval driven development

Anthropic’s first technical PM on token maxing, the jagged edge, and living in the future

Anthropic’s first technical PM on token maxing, the jagged edge, and living in the future

Dianne Penn, Head of Product for Anthropic’s AI Research and Labs teams, discusses Anthropic's journey, the evolving AI landscape, and the changing role of product management. She highlights the importance of adaptability, 'evals as the new PRDs,' and how Anthropic's focus on alignment and safety makes Claude a more effective 'thinking partner' by enabling it to push back on user ideas.

EDD: The Science of Improving AI Agents // Shahul Elavakkattil Shereef // Agents in Production 2025

EDD: The Science of Improving AI Agents // Shahul Elavakkattil Shereef // Agents in Production 2025

This talk introduces Eval-Driven Development (EDD) as a scientific alternative to 'vibe-based' iteration for improving AI agents. It covers quantitative evaluation (choosing strong end-to-end metrics, aligning LLM judges) and qualitative evaluation (using error and attribution analysis to debug failures), providing a structured framework for consistent agent improvement.