Russ d'Sa, CEO of LiveKit, recounts the company's unexpected journey from a pandemic-era open-source WebRTC project to becoming a crucial infrastructure provider for AI voice interfaces, most notably for OpenAI's ChatGPT. He details the serendipitous moments that led to this pivot and shares his vision for LiveKit as the nervous system for a multimodal AI future.
An in-depth look at building real-time, production-grade voice AI agents using the open-source Pipecat framework. This summary covers the core concepts of voice AI pipelines, the shift to speech-to-speech models like Gemini Live, and advanced techniques for managing latency, context, and turn-taking.
Sean DuBois (OpenAI, Pion) and Kwindla Hultman Kramer (Daily, Pipecat) argue that to build successful real-time AI applications, developers must start from the network layer up, prioritizing WebRTC over WebSockets to manage latency effectively and enable advanced features like interruption and state management.