AI progress is fundamentally driven by vast amounts of data and compute, rather than improvements in sample efficiency, creating a stark contrast with human learning. This essay explores the "black hole of data" powering AIs, quantifies the massive sample-efficiency gap between humans and machines, counters common objections, and discusses the implications for white-collar automation and future AI research.
This discussion with Finbarr Timbers reviews the evolution of frontier post-training recipes, highlighting the shift from simpler SFT-DPO-RL to complex multi-teacher on-policy distillation (MOPD). It covers the organizational challenges of building models like Olmo, the rise of synthetic data and reasoning-focused RL in DeepSeek, and the complexities of integrating expert teachers, while also exploring open questions on environments, specialized APIs, and career strategies in the rapidly changing AI landscape.
YC Visiting Partner Francois Chaubard and YC General Partner Ankit Gupta break down diffusion, the machine learning framework behind generative AI models like Sora and Midjourney. They discuss its core principles, trace its evolution from complex KL-divergence methods to the elegant simplicity of flow matching, and explore its vast applications beyond images, from protein folding to robotics, arguing it's a key component for future AI systems.
Mati Staniszewski, CEO of ElevenLabs, discusses the company's strategy for rapidly shipping research-grade AI. He covers their organizational structure of small, autonomous teams, a global and remote-first hiring philosophy, the transition from a creator-focused product to an enterprise platform, and the lessons learned in navigating complex licensing and scaling a go-to-market team.
OpenAI’s Chief Scientist, Jakub Pachocki, and Chief Research Officer, Mark Chen, discuss the research behind GPT-5, the push toward long-horizon reasoning, and the grand vision of an automated researcher. They cover how OpenAI evaluates progress beyond saturated benchmarks, the surprising durability of reinforcement learning, and the culture required to protect fundamental research while shipping world-class products.