Sep 17, 2026

The End of TCP for AI Clusters — John Ousterhout, Stanford

John Ousterhout argues that AI workloads are shifting from large data transfers to small, frequent coordination messages, making network latency (especially tail latency) a critical bottleneck. He explains why traditional protocols like TCP and RDMA are ill-suited due to sender-side congestion control and their byte-stream model. Ousterhout then introduces Homa, a new protocol designed from scratch with message-based communication, receiver-side congestion control, and switch priority queue utilization, demonstrating significant improvements in tail latency for AI applications.

Aug 27, 2026

LLM & AI Agent Benchmarks vs Reality: Why AI Applications Break

LLM leaderboard scores often don't reflect real-world performance. This video explains why and outlines a comprehensive approach to evaluate AI systems, focusing on the critical balance of accuracy, latency, and cost. It details model and system evaluation techniques, including handling different inference phases, workload shapes, and specific considerations for AI agents, emphasizing the need for realistic testing over generic benchmarks.

Jun 29, 2026

Frontier results, on device - RL Nabors, Arize

RL Nabors discusses the significant costs associated with using frontier AI models, covering security, latency, and financial implications. She introduces a framework for right-sizing AI solutions by leveraging smaller, task-specific models and Small Language Models (SLMs). The framework details how to prove task feasibility, establish success criteria with golden datasets, conduct capability evaluations (using tools like Phoenix), and select the most appropriate "Small And Good Enough" (SAGE) model. Nabors further demonstrates how prompt engineering, particularly few-shot prompting, and post-processing can close performance gaps with larger models, while advocating for continuous regression evaluations to maintain performance integrity. The overarching message is to "prototype big, deploy small" to optimize AI deployments.

May 20, 2026

The Latency Goldilocks Zone Explained

Rafael Borger and Daniel Wolbert from iFood discuss the engineering and product strategy behind ILO-Agent, their conversational AI for 200 million users. They cover hyper-personalized recommendation systems, the "Latency Goldilocks Zone" where AI responses can be too fast for users to trust, and the architectural challenges of building multi-channel agents for text and voice.

May 09, 2026

Voice AI: when is the "Her" moment? — Neil Zeghidour, Gradium AI

Neil Zeghidour, CEO of Gradium AI, deconstructs the gap between current voice AI and the "Her" ideal. He argues that while cascaded systems are practical, they are architecturally flawed for natural conversation. The future lies in full-duplex, speech-to-speech models that not only solve latency but also integrate deep paralinguistic understanding and overcome significant cost barriers.

Feb 18, 2026

Build Hour: Prompt Caching

Explore prompt caching to significantly reduce latency and costs for your AI applications. This guide breaks down the mechanics of KV caching, best practices for maximizing cache hits using `prompt_cache_key` and the Responses API, and real-world implementation insights from the agentic development platform, Warp.