How Contour Asset Management, a $6B fundamental long/short technology, media and telecom fund, measures the return on AI spend through live A/B tests of analyst research, a cost per useful insight metric, timing agents that resurface old ideas, and AI workflows across trading, operations and compliance.
Roland Gavrilescu, co-founder and CEO of Introspection and formerly at xAI, presents a blueprint for autoresearch in 2026 built on three ideas: the loop is the product, system distillation is the moat, and valued work per watt is the score to optimize. He illustrates these with the OpenClaw car-haggling loop, the concept of portable agent recipes, and a talent-sourcing agent example that shows how to spot patterns in traces, calibrate judges with humans, and A/B test taste in production.
This talk provides a practical framework for product managers to move beyond simple "vibe checks" to implement rigorous, data-driven evaluation for LLM-powered products. Using a live demo of a multi-agent AI trip planner, the speaker breaks down essential methodologies, including human feedback, code-based checks, and LLM-as-a-judge systems, and demonstrates how to iterate on both prompts and the evals themselves to ensure consistent quality and build user trust.