Machine learning operations ( mlops)

SimulationMaxxing: How Nubank ships agents 20× faster with simulations — Shreya Rajpal, Snowglobe

SimulationMaxxing: How Nubank ships agents 20× faster with simulations — Shreya Rajpal, Snowglobe

Nubank, serving 135 million customers, uses AI agents for support. The talk reveals how simulated data for evaluations (evals) has enabled them to ship AI agents 20x faster. By addressing the bottleneck of multi-turn, stateful eval data, Snowglobe's grounded simulations create realistic customer interactions, allowing rapid testing, derisking, and significant improvements in customer satisfaction and self-service rates, even for open-source model experimentation.

Plenary Talk 3​: Challenges and research opportunities for global hyperscale services

Plenary Talk 3​: Challenges and research opportunities for global hyperscale services

Jim Kleewein's talk outlines the immense challenges and critical research opportunities in building and operating global hyperscale services like Microsoft 365 and Azure. He emphasizes that at this scale, traditional approaches fail, necessitating a "new golden age of applied research" across areas like continuous availability, data management, security, and sustainability. Kleewein also discusses AI's powerful but limited role, stressing the ongoing need for human expertise, and highlights the ethical imperative to prevent failures that can have life-or-death consequences.