Martin Keen explains why bigger context windows can make AI agents less reliable. He introduces context rot, the four context failure modes of poisoning, distraction, confusion, and clash, and the context engineering fixes that work by removing information rather than adding it.
Snorkel AI and UC Berkeley's Sky Computing Lab show that a 4B Qwen3 model, trained with reinforcement learning for under $500 using the rLLM framework, beats its 235B sibling on expert-validated financial question answering built from SEC 10-K filings. The talk covers the FinQA dataset construction and verification, common failure modes of frontier models, transfer to multi-table and general tool-use tasks, and ablations showing that simple data and binary rewards outperform more elaborate setups, since the real bottleneck is disciplined tool use rather than reasoning depth.
Benjamin Clavié argues that the industry has mistakenly generalized from the "easy case" of coding agents to the much harder domain of general knowledge work. He highlights that code offers durable, explicit cues and narrow tasks, while knowledge work involves implicit meaning, contextual understanding, and agents needing to reconstruct intent. Clavié traces human solutions through "tool loops" and "organization loops," demonstrating how optimized tools and hierarchical agentic structures (like searcher agents generating memos) are essential to overcome performance ceilings and significantly bridge the "oracle gap" between AI and human knowledge workers.
Andrew Qu, Chief of Software at Vercel, details the year-long journey of building an internal data science agent, from its initial "awful" state to the release of Eve, a framework for agent development. He covers iterative architectural improvements, the critical "file system unlock" inspired by Claude Code, and the distillation of common queries into reusable "skills," culminating in a framework that simplifies the creation of specialized, company-specific AI agents.
Mercor CEO Brendan Foody elucidates the concept of RL environments, essential for training advanced AI agents. He breaks down their three core components—worlds, apps, and tasks—and details Mercor's evolution from crowdsourced data to expert-driven, "agentic" data. Foody underscores the indispensable role of human experts in defining frontier tasks and creating robust verifiers, exemplified by a real legal RL environment. He shares post-training results demonstrating significant performance gains with modest compute, discusses data pricing and quality, demystifies synthetic data, and explores future directions like ultra-long-horizon tasks and virtual co-workers. The talk emphasizes that data sets are becoming a critical moat for application-layer companies, enabling them to own their intelligence.
Anthropic's Gagan Bhat and Isabella Kai He discuss how agent harnesses must evolve rapidly to keep pace with fast-improving LLMs. They introduce Claude Managed Agents, an architecture that decouples the agent's 'brain' (reasoning) from its 'hands' (tool execution) to address issues like stale assumptions, latency, and reliability. This approach enables dynamic adaptation, secure tool execution, and features like 'dreaming' for self-improving agents and 'outcomes' for goal-oriented task completion, ultimately aiming to close the gap between model capabilities and product offerings.