Reinforcement learning

Aug 12, 2026

RL Environments Explained: How AI Agents Learn Real-World Work | Brendan Foody, Mercor

Mercor CEO Brendan Foody elucidates the concept of RL environments, essential for training advanced AI agents. He breaks down their three core components—worlds, apps, and tasks—and details Mercor's evolution from crowdsourced data to expert-driven, "agentic" data. Foody underscores the indispensable role of human experts in defining frontier tasks and creating robust verifiers, exemplified by a real legal RL environment. He shares post-training results demonstrating significant performance gains with modest compute, discusses data pricing and quality, demystifies synthetic data, and explores future directions like ultra-long-horizon tasks and virtual co-workers. The talk emphasizes that data sets are becoming a critical moat for application-layer companies, enabling them to own their intelligence.

Aug 10, 2026

Taking Reinforcement Learning Cross Datacenter — Nan Jiang, Modal

Nan Jiang introduces "Adam absorption" to revolutionize RL model synchronization. By exploiting finite precision serving and small Adam steps, less than 1% of served model weights actually change, allowing for 500MB patches instead of 500GB checkpoints. This enables a distributed "bulletin board" architecture, decoupling trainers from global rollout fleets and unlocking elastic, cross-region GPU capacity for RL.

Aug 08, 2026

Why Robotics Still Isn't Solved - But Could Be Soon | YC Paper Club

This Paper Club delves into the current state of robotics, addressing roadblocks like the sim-to-real gap and embodiment drift. Speakers present advancements in multi-scale memory for long-horizon tasks, self-supervised embodied reasoning, zero-shot dexterous manipulation via massive simulation, and the economic imperative of teleoperation-first robotics companies, concluding with optimizations for efficient, real-time World Action Models.

Aug 07, 2026

Local Models: Trust, Control, Optimization — Carter Abdallah, NVIDIA

A panel with Vincent Weisser (Prime Intellect), Lucas Atkins (Arcee AI), and Chris Alexiuk (NVIDIA) on why open models matter for trust, control, cost predictability, and data ownership. The discussion covers the distinction between trust and safety, post-training open models for specific harnesses and use cases, the open MDW license, the Linux-like role of open infrastructure alongside closed models, and predictions for open model capabilities and local AI adoption over the next year.

Aug 04, 2026

40 Trillion Tokens a Day (Yes, More Than OpenAI) | Lin Qiao, CEO of Fireworks

Lin Qiao, CEO of Fireworks, discusses why specialized AI models built on private company data are the future, contrasting them with general-purpose models. She argues for "open intelligence," revealing Fireworks processes over 40 trillion tokens daily from customized models, more than OpenAI's API. Qiao emphasizes the economic and strategic imperative for companies to own their specialized intelligence, advocating for open-sourcing by frontier labs like OpenAI and Anthropic, while detailing Fireworks' proprietary, quality-obsessed platform for tailored AI solutions.

Aug 01, 2026

Teaching AI to Find Real Vulnerabilities — David Brumley, Bugcrowd

David Brumley discusses the challenges and solutions for teaching AI models to hack, drawing parallels with human learning. He introduces a 'ladder of tasks' approach for reinforcement learning and addresses the critical flaw of traditional benchmarks: measurement difficulties with multiple vulnerabilities and 'reward hacking.' His team's 'Audit Task' uses deterministic graders and precision/recall metrics for open-world assessment. He demonstrates this with an in-depth case study on attacking Chrome's V8 engine, showcasing how advanced models achieve real zero-day exploits, and warns against 'benchmaxxing security' without robust, honest grading.