Rl environments

Can Open Models Keep Up With OpenAI and Anthropic? | Nathan Lambert

Can Open Models Keep Up With OpenAI and Anthropic? | Nathan Lambert

Nathan Lambert, AI researcher and founder of the nonprofit Trillium, joins Lisha Li to examine how durable the gap is between frontier closed labs like OpenAI and Anthropic and open-weight models. The discussion covers compute, data, and talent as the drivers of model progress, how much distillation really matters for Chinese open labs, the evolution of post-training toward RL environments, and why open research on RL environments and recursive self-improvement is needed to demystify what frontier systems can and cannot do.

Better AI Starts With Better Verification

Better AI Starts With Better Verification

Halluminate, a Y Combinator S25 company, builds RL environments and benchmarks that teach AI models to do non-coding knowledge work, starting with financial services. In this fireside with YC Partner Jon Xu, co-founders Jerry Wu and Wyatt Marshall trace their pivot from evals to training environments, explain why verification quality matters more than task volume, and outline a roadmap toward multi-agent simulated companies. They also address reward hacking, alignment as an industry-wide responsibility, and what they look for in new hires.

Going In Deep On Data | YC Paper Club

Going In Deep On Data | YC Paper Club

Three experts discuss the evolving landscape of data in AI, covering its critical role in model performance, challenges in sourcing and evaluating expert data, the development of advanced data generation techniques for diffusion models, and the complexities of multilingual pre-training.

RL Environments Explained: How AI Agents Learn Real-World Work | Brendan Foody, Mercor

RL Environments Explained: How AI Agents Learn Real-World Work | Brendan Foody, Mercor

Mercor CEO Brendan Foody elucidates the concept of RL environments, essential for training advanced AI agents. He breaks down their three core components—worlds, apps, and tasks—and details Mercor's evolution from crowdsourced data to expert-driven, "agentic" data. Foody underscores the indispensable role of human experts in defining frontier tasks and creating robust verifiers, exemplified by a real legal RL environment. He shares post-training results demonstrating significant performance gains with modest compute, discusses data pricing and quality, demystifies synthetic data, and explores future directions like ultra-long-horizon tasks and virtual co-workers. The talk emphasizes that data sets are becoming a critical moat for application-layer companies, enabling them to own their intelligence.

Data and Environment Curation for Post-Training LLMs — Mahesh Sathiamoorthy, Bespoke Labs

Data and Environment Curation for Post-Training LLMs — Mahesh Sathiamoorthy, Bespoke Labs

Mahesh Sathiamoorthy of Bespoke Labs argues that high-quality data and curated RL environments are the true bottlenecks for post-training LLMs, especially for building reliable, autonomous agents. He grounds this in experiences with OpenThoughts, a reasoning dataset, highlighting counterintuitive lessons like the importance of diverse reasoning traces and the fact that stronger teachers aren't always best. A key takeaway, reinforced by their Curator tooling, is that a disciplined curation stack is essential for transforming base models into capable, post-trained agents for real-world applications like credit card compliance.