MasterClass engineers Mandar and Priya explain how they built NCX, a multi-agent AI tutoring system, and the CoreWeave-based tracing, MCP server, and AgentOps loop that keeps it reliable and self-improving in production.
Fuad Ali from Arize AI explains why voice agents are uniquely challenging to debug, as critical issues like latency, interruptions, and tone are invisible in text logs. He presents a solution combining unified audio, transcript, and trace views with OpenInference semantic conventions and audio-native evaluations. The talk culminates in a vision for self-healing voice agents through automated observation, evaluation, and improvement cycles, demonstrating how to achieve robust, continuously improving AI voice systems.
Jason Ward, FinOps for AI lead at C.H. Robinson, shares insights on managing AI costs for enterprises and startups. He details a FinOps strategy centered on comprehensive tagging, leveraging AI for observability (e.g., detecting prompt bloat and context starvation), and pioneering advanced metrics like "cost per thought" and "reasoning ratio" to optimize complex AI workloads beyond simple discrete tasks.
Sherwood Callaway of 11X discusses the architecture of "Alice," an AI Sales Development Representative. He covers the practical decision to use LangGraph for its reliability in production, the challenges of infrastructure and observability when using hosted agent platforms, and their methodology for running Evals to mitigate hallucinations by comparing generated content against source data.