How Physical AI Learns Across Language, Video and Action — Ming-Yu Liu

Ming-Yu Liu from NVIDIA details Cosmos 3, a novel omnimodal world model capable of video description, generation, and robot action synthesis. He elaborates on its architecture, which integrates a vision language model with a bidirectional diffusion generator. Liu defines world models as a collection of tools comprising forward dynamics, inverse dynamics, and policy, trained synergistically. The discussion covers learning robot skills from human video, the crucial role of neural simulators like Cosmos Dreams for policy verification and accelerating development in robotics, and the importance of safety for future humanoid robots. The episode concludes by outlining the Super, Nano, and Edge model sizes and NVIDIA's commitment to open-sourcing Cosmos's code, models, and data.