![[Full Workshop] Building Metrics that actually work — David Karam, Pi Labs (fmr Google Search)](https://img.youtube.com/vi/jxrGodnopHo/maxresdefault.jpg)
This workshop, led by former Google product directors, introduces a methodology for building reliable and tunable evaluation metrics for LLM applications. It details how to create granular 'scoring systems' that break down complex evaluations into simple, objective signals, and then use these systems for model comparison, prompt optimization, and online reinforcement learning.
Developing general-purpose robots requires a shift from specialized, single-task systems to broad foundation models. This is achieved through a combination of large-scale, diverse, real-world data collection and a specific training methodology: pre-training on all available data and then fine-tuning on a curated, high-quality subset of demonstrations. This recipe, combined with architectural innovations to preserve the capabilities of Vision-Language Model (VLM) backbones, enables robots to perform complex, long-horizon tasks, generalize to unseen environments, and respond to open-ended human instructions.