This talk addresses the unreliability of current AI agents, arguing that prompting is insufficient. It posits that Reinforcement Learning (RL) is the most promising solution, delving into the mechanisms of RLHF and RLVR. The core challenge identified is 'reward hacking', and the discussion explores future directions to overcome it, such as RLAIF, data augmentation, and the development of interactive, online models that can learn in real-time.
Julien Launay, CEO of Adaptive ML, discusses the evolution of Large Language Model (LLM) training, detailing the critical shift from pre-training to post-training with Reinforcement Learning (RL). He explains the nuances of RL feedback mechanisms (RLHF, RLEF, RLAIF), the role of synthetic data, and how his company provides the "RLOps" tooling to make these powerful techniques accessible to enterprises. The conversation also explores the future of AI, including scaling beyond data limitations and the path to a "spiky" AGI.
Edwin Chen, CEO of Surge AI, discusses the critical role of high-quality human data in training frontier models, the flaws in current evaluation benchmarks like LMSys and IF-Eval, the future of complex RL environments, and why he bootstrapped Surge to over $1 billion in revenue.