Browser agents for financial data behind logins and anti-bot protection. Covers Ramp, Bill, and Stripe case studies, why cloud browsers get blocked, how Browserbase forked Chromium for reliable data-center use, and how Stagehand's act, extract, and observe primitives keep automations resilient to site changes.
Vinoo Ganesh, CEO of Kepler, explains how the company builds a verifiable financial research platform. Models cite numbers but never write them. Deterministic code performs the math, a citation registry built from more than 30 million SEC documents anchors every figure, and a post-processing filter strips anything untraceable. The talk covers hybrid search and re-ranking on Turbopuffer, tenant isolation, and why plain RAG produced thousands of wrong numbers in evaluations.
Janie Gu of Google DeepMind and Kaggle explains why most AI benchmarks are unreliable: they don't reflect real work, they are costly to keep current, and their scores are hard to verify. Drawing on a reproduction of 65 industry benchmarks, she details the failure modes and how Kaggle is building auditable, containerized evaluation infrastructure. Suyeol Yun of LinqAlpha then presents MandateBench, which tests whether models can carry out buy-side equity analyst work from a fund mandate to a full investment memo.
Laurie Voss, head of developer relations at Arize AI and co-founder of npm, argues that coding agents have made writing code nearly free, so the bottleneck keeps moving first to code review, then to operations and security. What agents can't take over is discovering what people actually need and defining what 'good' means. He introduces the product engineer as the role that remains, examines the collapse in junior engineering hiring, and offers practical advice for engineering leaders.
Sofia Jakovcevic, a forward deployed engineer at Arize AI, argues that agents fail at runtime rather than in code, so regressions from a small prompt change can hide behind an improved metric. She walks through a five-step regression loop built on production traces, including dataset curation, LLM-as-a-judge evaluators, baseline experiments, comparisons, and CI merge gates, then shows how coding agents with access to traces can close the loop themselves.
Bridgewater Associates' Applied AI and product engineering leads explain how they built the Vault, a governed repository of more than 100 million unstructured documents, and why humans and agents need different things from the same search system. They cover a hand-built benchmark that lifted precision at 10 from about 60% to over 90%, the filtering and scaling choices behind it, and the shift toward proactive agents that bring investors the topics most deserving of their attention.