Anthropic’s first technical PM on token maxing, the jagged edge, and living in the future
Dianne Penn, Head of Product for Anthropic’s AI Research and Labs teams, discusses Anthropic's journey, the evolving AI landscape, and the changing role of product management. She highlights the importance of adaptability, 'evals as the new PRDs,' and how Anthropic's focus on alignment and safety makes Claude a more effective 'thinking partner' by enabling it to push back on user ideas.
Dianne Penn, Head of Product for Anthropic’s AI Research and Labs teams, shares insights into Anthropic’s rapid growth, the evolution of AI, and the changing role of product management. She describes Anthropic’s early days as a lean startup in 2023, with a strong mission-driven culture and a focus on bottoms-up experimentation, epitomized by projects like ‘Golden Gate Claude.’ The company’s inflection points included the development of Opus 3, which established Claude’s frontier model capabilities and coding prowess, and Opus 45, which, alongside products like Claude Code, showcased the synergy between frontier models and product experiences.
Penn highlights the unique challenges and opportunities of operating within the ’exponential curve’ of AI advancement. Adaptability and first-principles thinking are crucial, as new, discontinuous ’emerging capabilities’ can appear unexpectedly. She emphasizes that there’s significant ‘product overhang’ and ‘user overhang,’ meaning much potential remains undiscovered even with current models. Experimentation, rather than just token spend, is the key output, fostered by communal discovery and internal ‘working in public’ channels.
Anthropic Labs plays a critical role in incubating ‘discontinuous large bets’ that may not fit into the core roadmap, such as Claude Code, Skills, and Claude Design. The lab’s success stems from strong vision on themes, flexible approaches to prototypes, and a culture of zero-to-one experimentation with small, self-driven teams.
Discussing the research role, Penn explains that researchers have a bold, long-term vision for AI (e.g., AI navigating screens) alongside iterative model improvements. Product managers act as crucial bridges, translating vague user feedback (like ‘Claude hallucinates’) into actionable, detailed insights for researchers through meticulously crafted ’evals.’ She famously states, “evals are the new PRDs” — a shift from traditional product requirements to data-driven assessments of model performance on specific user pain points, akin to ’test-driven development’ for PMs.
Regarding the evolution of the product role, Penn notes that Anthropic’s hiring loop has remained consistent, prioritizing first-principles thinking, deep user understanding (sweating tokens as much as pixels), and hands-on engagement with the technology. She stresses that product leaders must remain hands-on, actively building and shipping to maintain their understanding of the rapidly evolving models. Finding joy in AI work comes from communal discovery and solving personal problems, rather than solitary experimentation.
Penn shares how she uses Claude for personal growth, leveraging it as a personalized coach for ‘crucial conversations’ and brainstorming. She believes that AI should augment human thinking, acting as a ‘sparring partner’ that pushes back, rather than just agreeing. This capability for Claude to challenge users is rooted in Anthropic’s alignment and safety research, where a ‘constitution’ makes Claude a more useful and interesting conversationalist, allowing for better outcomes than a merely compliant AI. She also touches on the current limitations of AI writing, noting that Anthropic is actively working to improve Claude’s tone and character.
Looking to the future, Penn believes human brains will remain most valuable for ‘judgment’ (an accumulation of nuance and experience), persistence, proactivity, and subject matter expertise in fields yet to be fully transformed by AI, like biology. For parents, she advises fostering curiosity, persistence, and an ‘inner voice’ in children, echoing the importance of maintaining one’s own perspective even with advanced AI tools. To combat burnout in a fast-paced environment, Penn credits Anthropic’s culture of radical ownership, team collaboration, and a ‘hive mind’ approach, emphasizing mutual support and low-ego contributions.