MasterClass engineers Mandar and Priya explain how they built NCX, a multi-agent AI tutoring system, and the CoreWeave-based tracing, MCP server, and AgentOps loop that keeps it reliable and self-improving in production.
Maggie Konstanty, AI Product Manager at Prosus, provides a candid look into the realities of LLM evaluation in production. She argues that standard metrics like accuracy are misleading and advocates for a culture of continuous, goal-oriented evaluation focused on deep failure analysis and understanding real user behavior, asserting that mature teams inevitably build custom tooling to meet their specific needs.