Reliability

How to build an AI-Native Health Company — Dan Feng, Maven Clinic

How to build an AI-Native Health Company — Dan Feng, Maven Clinic

Dan Feng outlines Maven Clinic's transformation into an AI-native company, driven by the realization that "building is cheap and arguing is expensive" in the AI era. This shift has reshaped planning to short, iterative sprints, revolutionized software development with AI coding tools, forced adaptive changes in code review processes, and necessitated a nuanced, multi-layered approach to ensuring reliability in generative AI systems.

Your Agent Didn't Fail. Your Harness Did. — Vinoth Govindarajan, OpenAI

Your Agent Didn't Fail. Your Harness Did. — Vinoth Govindarajan, OpenAI

Vinoth Govindarajan's talk addresses critical 'harness failures' in AI agents, arguing these, rather than model errors, are the root cause of most production incidents. He introduces the core contract: 'A model proposes, the harness commits, and a receipt proves it,' and outlines five key boundaries (state ownership, ordering, deadlines, authority, user-visible proof) that lead to failures like silent success and incomplete reality. The talk culminates in a practical 'run receipt audit' with five questions to diagnose and ensure reliable agent behavior.

Perception Agents — Antje Barth, Amazon AGI Lab

Perception Agents — Antje Barth, Amazon AGI Lab

Antje Barth of Amazon AGI Lab discusses the architectural gap in current AI agents, which excel at individual tasks but fail at complex, end-to-end workflows due to a lack of reliability and contextual understanding. She introduces "Perception Agents"—AI systems that see, reason, and act on computers like humans, using visual and multimodal input to enable reliable collaboration and close the perception-action loop, highlighting new open-source tools for annotation and verification.

In the Land of AI Agents, the Verifiers Are King — Tariq Shaukat, Sonar

In the Land of AI Agents, the Verifiers Are King — Tariq Shaukat, Sonar

This talk addresses the critical challenge of verification in AI agent development, moving beyond generation to ensure correctness. It highlights the problem of "AI slop" and the "productivity paradox" of AI coding agents, where initial velocity gains are offset by increased technical debt and quality issues. The speaker introduces the AC/DC (Agent-Centric Development Cycle) framework comprising three stages: Guide (providing context and constraints), Verify (zero-trust, multi-layered verification using both algorithmic and agentic methods), and Solve (active code maintenance to control technical debt). This systems-level approach, integrating verification across agentic, CI, and code maintenance loops, significantly reduces issues and transforms AI into a reliable enterprise asset.

Don't Ship Skills Without Evals — Philipp Schmid, Google DeepMind

Don't Ship Skills Without Evals — Philipp Schmid, Google DeepMind

Philipp Schmid from Google DeepMind emphasizes the critical, often-overlooked need for rigorous evaluation of AI agent skills. He argues that shipping skills without testing is akin to deploying code without unit tests, leading to unreliable agent behavior. The talk covers what defines an agent skill, strategies for writing effective and correctly triggering skills, and a practical guide to building lightweight evaluation harnesses to catch failures proactively.

Harnesses in AI: A Deep Dive — Tejas Kumar, IBM

Harnesses in AI: A Deep Dive — Tejas Kumar, IBM

A deep dive into AI harnesses, explaining how to build a programmatic environment around an LLM agent to ensure reliability without prompt engineering. The talk demonstrates building a harness for a browser agent to reliably log in and upvote a post on Hacker News using GPT-3.5 Turbo.