Llms

Building Closed-Loop Evals for a Multimodal Agent at Scale — Soumya Gupta & Jai Chopra, Uber

Building Closed-Loop Evals for a Multimodal Agent at Scale — Soumya Gupta & Jai Chopra, Uber

This talk details how Uber Eats designed and implemented a multimodal AI agent to enhance food photography for independent merchants, addressing challenges like maintaining authenticity, merchant brand, and marketplace diversity while operating at scale. It covers the intricate evaluation strategies for routing and image editing agents, including continuous learning loops, managing drift, countering reward hacking, and balancing creative freedom with rigid safety guardrails. The speakers explain how they built a closed feedback loop combining offline human labeling, internal dogfooding, and online production signals to ensure robust and adaptive performance.

Coding Agents Are Secretly General Agents

Coding Agents Are Secretly General Agents

Jay Hack, head of AI at ClickUp, discusses the evolution of AI from early computer vision to generalist coding agents, highlighting how 'positive transfer' makes coding an 'AGI-complete' domain. He delves into the brutal economics of AI startups facing foundation model giants, ClickUp's strategy for convergence and first-party data as a moat, and the challenges of verifiability and catastrophic forgetting. The conversation also explores LLMs at the scientific frontier, the 'car wash test' revealing limits of world models, and speculative future applications like LLM resorts and game integration.

How Supabase Became One Of The Fastest Growing DevTool Companies In The World

How Supabase Became One Of The Fastest Growing DevTool Companies In The World

Supabase CEO Paul Copplestone details how a frustrating Firebase migration sparked an open-source side project that grew into a decacorn. He explains the strategic choice of PostgreSQL and open source, the evolution of developer experience to achieve 5-second time-to-value, and the transformative impact of AI agents, which now launch millions of Supabase databases monthly. The conversation also covers the challenges and advantages of a fully distributed workforce and the future bet on 'self-driving databases' to address the complex 'operate' stage of AI-driven development.

Why Agentic Systems Need Ontologies — Frank Coyle, UC Berkeley

Why Agentic Systems Need Ontologies — Frank Coyle, UC Berkeley

Frank Coyle discusses the inherent flaws of purely probabilistic LLMs in agentic systems and proposes a neuro-symbolic solution. He advocates for wrapping LLM tool-use loops with external ontologies and validation mechanisms (like RDFS, OWL, and Pydantic) to provide logical guardrails, preventing common, critical errors that probabilistic reasoning alone cannot reliably avoid.

2026 State of AI Engineering — Barr Yaron, Amplify Partners

2026 State of AI Engineering — Barr Yaron, Amplify Partners

Barr Yaron's 2026 AI engineering survey reveals key trends: audio and image generation are rapidly gaining traction, while cost is now a primary engineering constraint. Agents are evolving to take actions within systems, but control mechanisms remain primitive. Evaluation (eval) is still the top infrastructure challenge. AI positively impacts job satisfaction and experimentation but also raises concerns about technical skill erosion and non-developers shipping code, fundamentally changing engineering culture. Predictions include a likely AGI declaration within five years and a shift away from Transformers as state-of-the-art.

In the Land of AI Agents, the Verifiers Are King — Tariq Shaukat, Sonar

In the Land of AI Agents, the Verifiers Are King — Tariq Shaukat, Sonar

This talk addresses the critical challenge of verification in AI agent development, moving beyond generation to ensure correctness. It highlights the problem of "AI slop" and the "productivity paradox" of AI coding agents, where initial velocity gains are offset by increased technical debt and quality issues. The speaker introduces the AC/DC (Agent-Centric Development Cycle) framework comprising three stages: Guide (providing context and constraints), Verify (zero-trust, multi-layered verification using both algorithmic and agentic methods), and Solve (active code maintenance to control technical debt). This systems-level approach, integrating verification across agentic, CI, and code maintenance loops, significantly reduces issues and transforms AI into a reliable enterprise asset.