This talk explores methods for optimizing expressive, multi-modal policies, such as those based on flow-matching, with off-policy reinforcement learning. The speaker presents two novel algorithms, FQ-RL and CAM, designed to overcome the instability of backpropagation through multi-step generative models, enabling effective online self-improvement and adaptation for robotic manipulation tasks.
Nikolaus West, CEO of Rerun, explains how their data logging and visualization platform, built on an Entity Component System (ECS) inspired by gaming, is unlocking new capabilities in physical AI. He discusses the rapid progress in robot manipulation through imitation learning, the gap between impressive demos and real-world products, and the critical need for better data tooling to handle complex, multi-rate sensor data in robotics and AR/VR.
Tony Zhao and Cheng Chi, co-founders of Sunday Robotics, discuss the state of AI robotics, framing it as being between a "GPT moment" (the core technology is known) and a "ChatGPT moment" (a scalable consumer product). They detail the key research, data collection innovations, and full-stack engineering required to build Memo, their general-purpose home robot designed to free humanity from chores.
A reflection on Richard Sutton's "Bitter Lesson," arguing that while his critique of LLMs' inefficiency and lack of continual learning is valid, imitation learning is a complementary and necessary precursor to true reinforcement learning, much like fossil fuels were to renewable energy.
Developing general-purpose robots requires a shift from specialized, single-task systems to broad foundation models. This is achieved through a combination of large-scale, diverse, real-world data collection and a specific training methodology: pre-training on all available data and then fine-tuning on a curated, high-quality subset of demonstrations. This recipe, combined with architectural innovations to preserve the capabilities of Vision-Language Model (VLM) backbones, enables robots to perform complex, long-horizon tasks, generalize to unseen environments, and respond to open-ended human instructions.