World models

How Physical AI Learns Across Language, Video and Action — Ming-Yu Liu

How Physical AI Learns Across Language, Video and Action — Ming-Yu Liu

Ming-Yu Liu from NVIDIA details Cosmos 3, a novel omnimodal world model capable of video description, generation, and robot action synthesis. He elaborates on its architecture, which integrates a vision language model with a bidirectional diffusion generator. Liu defines world models as a collection of tools comprising forward dynamics, inverse dynamics, and policy, trained synergistically. The discussion covers learning robot skills from human video, the crucial role of neural simulators like Cosmos Dreams for policy verification and accelerating development in robotics, and the importance of safety for future humanoid robots. The episode concludes by outlining the Super, Nano, and Edge model sizes and NVIDIA's commitment to open-sourcing Cosmos's code, models, and data.

AI Is Missing This Data

AI Is Missing This Data

David Fattal, CTO of Leia Inc., explains why stereo data is crucial for developing robust AI world models that can accurately understand 3D geometry and interact with the physical world. He contrasts it with monocular data, highlighting stereo's superior ability to ground AI in reality. Fattal introduces Leia's technology for glasses-free 3D displays on standard devices, predicting a "device-data flywheel" where accessible 3D hardware drives user-generated stereo content, providing essential data for AI advancement, a crucial differentiator for future AI companies.

Why World Models Could Change Robotics, 3D, and Creativity

Why World Models Could Change Robotics, 3D, and Creativity

World Labs co-founders discuss Atlas, their frontier world model centered on "new view prediction." This paradigm unifies 3D generation and reconstruction, enabling precise spatial understanding from sparse inputs. The conversation covers its multimodal architecture, the shift from dense to sparse reconstruction, and its profound implications for creative industries, architecture, and particularly robotics, where it addresses critical data scarcity challenges for simulation and policy learning.

SOTA Generative Media Panel — Dumitru Erhan, Shane Gu & Nicole Brichtova, Google DeepMind

SOTA Generative Media Panel — Dumitru Erhan, Shane Gu & Nicole Brichtova, Google DeepMind

This detailed discussion with Google AI researchers highlights the latest generative video models (NanoBanana 2 Lite, Gemini Omni Flash APIs) and their applications, including video editing, education, and localization. The conversation critically examines whether language is an adequate intermediate representation for AI, especially for sensory experiences like audio and taste, where human vocabulary is limited. It exposes the unreliability of human preferences in evaluating AI-generated content, citing the "Instagram filter effect" and "wedding ring" reward hacking as examples. The researchers emphasize the importance of high-quality, real-world data and the critical role of forward-deployed engineers in bridging the gap between cutting-edge research and practical applications.

The Next Medium: Why Real-Time Interactive Video Changes Everything — Ahmed Ahres, Reactor

The Next Medium: Why Real-Time Interactive Video Changes Everything — Ahmed Ahres, Reactor

Ahmed Ahres argues that real-time interaction fundamentally changes the medium, not just its speed, especially in generative AI. He defines "world models" as interactive, effectively infinite, and steerable video, drawing parallels with GPS and camera viewfinders. This paradigm shift unlocks new forms of control, intelligent advertising, programmable worlds for robotics and education, and advanced live avatars. He highlights the critical infrastructure challenges of streaming pixels, managing stateful sessions, and achieving global sub-100ms latency, emphasizing that batch infrastructure is unsuitable for real-time applications. Evaluation of consistency for these models remains an unsolved problem.

Waymo Co-CEO Dmitri Dolgov: The Demo Is Only 1% Of The Work

Waymo Co-CEO Dmitri Dolgov: The Demo Is Only 1% Of The Work

Waymo co-CEO Dmitri Dolgov outlines seven crucial lessons from fifteen years of developing and scaling the Waymo Driver, the world's most advanced physical AI. He details the unique challenges of physical AI compared to digital, emphasizing the critical role of reliability, strategic technology choices, continuous innovation through foundation models, structure-augmented learning, high-fidelity simulation, AI flywheels, and robust evaluation frameworks to achieve superhuman safety and build trust in real-world autonomous systems.