Reinforcement learning

Aug 08, 2026

Why Robotics Still Isn't Solved - But Could Be Soon | YC Paper Club

This Paper Club delves into the current state of robotics, addressing roadblocks like the sim-to-real gap and embodiment drift. Speakers present advancements in multi-scale memory for long-horizon tasks, self-supervised embodied reasoning, zero-shot dexterous manipulation via massive simulation, and the economic imperative of teleoperation-first robotics companies, concluding with optimizations for efficient, real-time World Action Models.

Aug 04, 2026

40 Trillion Tokens a Day (Yes, More Than OpenAI) | Lin Qiao, CEO of Fireworks

Lin Qiao, CEO of Fireworks, discusses why specialized AI models built on private company data are the future, contrasting them with general-purpose models. She argues for "open intelligence," revealing Fireworks processes over 40 trillion tokens daily from customized models, more than OpenAI's API. Qiao emphasizes the economic and strategic imperative for companies to own their specialized intelligence, advocating for open-sourcing by frontier labs like OpenAI and Anthropic, while detailing Fireworks' proprietary, quality-obsessed platform for tailored AI solutions.

Aug 01, 2026

Teaching AI to Find Real Vulnerabilities — David Brumley, Bugcrowd

David Brumley discusses the challenges and solutions for teaching AI models to hack, drawing parallels with human learning. He introduces a 'ladder of tasks' approach for reinforcement learning and addresses the critical flaw of traditional benchmarks: measurement difficulties with multiple vulnerabilities and 'reward hacking.' His team's 'Audit Task' uses deterministic graders and precision/recall metrics for open-world assessment. He demonstrates this with an in-depth case study on attacking Chrome's V8 engine, showcasing how advanced models achieve real zero-day exploits, and warns against 'benchmaxxing security' without robust, honest grading.

Jul 31, 2026

Learning on the Job: The Future of Post-Training — Raymond Feng, Applied Compute

Raymond Feng presents Applied Compute's approach to training custom AI models that learn "on the job" using reinforcement learning. He details the evolution from controlled Q&A to synthetic environments, highlighting the core GRPO-style loop. A major focus is tackling the challenges of environment fidelity and "reward hacking" in simulated settings. The discussion then moves to the complexities of training directly within real-world enterprise harnesses, addressing issues like non-replayability and off-policy data. Feng concludes by outlining frontier research in self-distillation, automated data pipelines, and qualitative feedback, envisioning a future where models continuously learn and self-evaluate from every interaction, making "experience the dominant medium of improvement."

Jul 31, 2026

Data and Environment Curation for Post-Training LLMs — Mahesh Sathiamoorthy, Bespoke Labs

Mahesh Sathiamoorthy of Bespoke Labs argues that high-quality data and curated RL environments are the true bottlenecks for post-training LLMs, especially for building reliable, autonomous agents. He grounds this in experiences with OpenThoughts, a reasoning dataset, highlighting counterintuitive lessons like the importance of diverse reasoning traces and the fact that stronger teachers aren't always best. A key takeaway, reinforced by their Curator tooling, is that a disciplined curation stack is essential for transforming base models into capable, post-trained agents for real-world applications like credit card compliance.

Jul 31, 2026

Reinforcement Learning without Verifiable Rewards — Will Brown, Prime Intellect

Will Brown's talk addresses the critical challenge of applying Reinforcement Learning to real-world tasks where verifiable rewards are absent. He outlines how Primordial AI tackles this by leveraging environments as the core anchor for building reward signals. Key strategies include using LLMs as "judges," grounding tasks in production traces or document corpora, and employing "reverse direction" techniques to generate training data. Brown also details methods for calibrating task difficulty, identifying reward hacking, and fostering continual learning by treating model optimization as a science, emphasizing the use of compute to refine environmental signals and abstract human expertise.