Evals

Build Hour: Voice Agents

Build Hour: Voice Agents

A deep dive into building sophisticated voice agents using OpenAI's Realtime API and Agents SDK. The session covers architectural patterns like chained vs. end-to-end models, the use of multi-agent systems with handoffs for specialized tasks, and best practices for production including debugging with traces, implementing guardrails, and creating robust evaluations.

Build Hour: Reinforcement Fine-Tuning

Build Hour: Reinforcement Fine-Tuning

A deep dive into Reinforcement Fine-Tuning (RFT), covering how to set up tasks, design effective graders, and run efficient training loops to improve model reasoning, based on a live demonstration from OpenAI's Build Hours.

Perceptual Evaluations: Evals for Aesthetics — Diego Rodriguez, Krea.ai

Perceptual Evaluations: Evals for Aesthetics — Diego Rodriguez, Krea.ai

KREA.ai's cofounder Diego Rodriguez discusses the critical failure of current AI evaluation metrics in understanding human perception and aesthetics, advocating for a new paradigm of personalized, perceptually-aware evals.

AI Agent Development Tradeoffs You NEED to Know

AI Agent Development Tradeoffs You NEED to Know

Sherwood Callaway of 11X discusses the architecture of "Alice," an AI Sales Development Representative. He covers the practical decision to use LangGraph for its reliability in production, the challenges of infrastructure and observability when using hosted agent platforms, and their methodology for running Evals to mitigate hallucinations by comparing generated content against source data.