Dr. Sarah Gebauer presents a clinical framework for deploying AI agents in healthcare, drawing a powerful analogy between AI agents and medical residents. She outlines the critical risks, validation strategies, and post-deployment monitoring required to make agents useful, safe, and credible in high-stakes clinical environments.
Brendan Foody, CEO of Mercor, discusses the critical role of AI evaluations (evals) in model improvement, detailing how his company achieved unprecedented growth by supplying high-skilled experts to top AI labs. He explores the shift to Reinforcement Learning from AI Feedback (RLAIF), the future of work in an AI-driven economy, and why he believes the path to AGI is paved with better evals, not just more data.
Andrew Rabinovich, CTO and Head of AI at Upwork, details their strategy for building AI agents for digital work. He introduces a custom reinforcement learning approach called RLEF (Reinforcement Learning from Experience), explains why digital work marketplaces are ideal training grounds, and shares his vision for a future where AI delivers finished projects, orchestrated by a meta-agent named Uma.
Jesse Zhang, co-founder and CEO of Decagon, discusses how their AI agents are revolutionizing customer service for large enterprises by replacing mundane human labor. He covers their go-to-market strategy, the importance of a hardworking in-office culture, his journey as a second-time founder, and the future of an agentic world where AIs interact on behalf of companies and consumers.
Learn how W&B Weave's online evaluations enable real-time monitoring of AI applications in production, allowing teams to track performance, catch failures, and iterate on quality over time using LLM-as-a-judge scores.
This summary covers a deep dive into the paper "A Survey of Context Engineering for Large Language Models". The discussion reframes the conversation from simple prompt engineering to a more systematic approach of building information environments for LLMs. It explores the foundational components of context engineering—generation, processing, and management—and their application in advanced systems like Retrieval-Augmented Generation (RAG), memory, tool use, and multi-agent systems.