An exploration of Waymo's research into EMMA, an End-to-End Multimodal Model for Autonomous Driving. This summary details how foundation models like Gemini are being adapted to create a single, generalizable system that processes raw sensor data directly into driving decisions, aiming to solve the long-tail problem and improve scalability. It also covers the use of generative AI for advanced sensor simulation and model evaluation.
Edwin Chen, CEO of Surge AI, discusses the critical role of high-quality human data in training frontier models, the flaws in current evaluation benchmarks like LMSys and IF-Eval, the future of complex RL environments, and why he bootstrapped Surge to over $1 billion in revenue.
Developing general-purpose robots requires a shift from specialized, single-task systems to broad foundation models. This is achieved through a combination of large-scale, diverse, real-world data collection and a specific training methodology: pre-training on all available data and then fine-tuning on a curated, high-quality subset of demonstrations. This recipe, combined with architectural innovations to preserve the capabilities of Vision-Language Model (VLM) backbones, enables robots to perform complex, long-horizon tasks, generalize to unseen environments, and respond to open-ended human instructions.