Data and Environment Curation for Post-Training LLMs — Mahesh Sathiamoorthy, Bespoke Labs

Mahesh Sathiamoorthy, co-founder and CEO of Bespoke Labs, addresses the critical challenge of data and environment curation for post-training LLMs, particularly for autonomous agents. Bespoke Labs, an applied data research lab, aims to provide high-quality data and RL environments to enterprises and frontier labs. Their work includes the “Curator” tool for synthetic data, “Bespoke Stratos” which evolved into the “OpenThoughts” reasoning dataset, and contributions to “Terminal Bench”, alongside developing custom RL environments and post-training services. A core philosophy is for data curators to “walk in the shoes of the researcher” to effectively move model metrics. The evolution of AI has shifted from evaluating what models “know” (e.g., STEM/humanities benchmarks) to what agents can “do” (e.g., swb bench, Terminal Bench), emphasizing autonomous operation over long durations. The primary obstacle to long-term agent autonomy is reliability, and post-training, especially through reinforcement learning, is a powerful mechanism to enhance this. Mahesh asserts that for post-training, whether supervised fine-tuning (SFT) or RL, data is the bottleneck, with RL environments viewed as “data in a different shape.” While compute, models, and infrastructure are maturing, access to high-quality data and RL environments remains the struggle for many. The “OpenThoughts” project, a collaboration with Stanford, UC Berkeley, and Udub, addressed the community’s lack of high-quality reasoning data. They developed a scalable curation recipe, demonstrated by improving benchmarks like Amy and LiveCodeBench, and gained significant recognition. The “OpenThoughts” pipeline involves selecting source questions, mixing and filtering them, and generating multiple answers using teacher models, systematically refining the process through ablations. Key counterintuitive lessons from “OpenThoughts” include: sampling multiple answers per question (e.g., 16 times) significantly aids by providing diverse reasoning traces during fine-tuning, and stronger teacher models are not always the best. Similarly, “OpenThoughts Agents” extended this curation methodology to agent training, again observing that sampling multiple answers and non-top-tier teachers can be more effective, while synthetic rewriting and task augmentation were less successful. For enterprise applications, SFT often provides substantial gains, with RL being more compute-intensive for the “last few percentages” of improvement. A concrete example of post-training in production is Intuit’s Credit Karma app. The challenge was enabling a model to recommend credit cards with compliant reasoning without blowing up latency due to complex rule-based prompting. Bespoke Labs developed a specific curation recipe that used “tags” in the data to guide the model’s focus on the form of the response rather than specific numerical details (which could be hallucinated, like “0% APR”). This approach dramatically improved compliance metrics, reduced latency, increased throughput, and allowed Intuit to “own the model” and manage costs effectively against increasingly expensive frontier models. Bespoke Labs’ “Curator” tool facilitates this process, allowing users to define prompts from datasets or logs, generate responses, and fine-tune models with integrations to services like Tinker and Fireworks. Mahesh concludes by outlining a comprehensive “curation stack” or reference architecture for building and post-training agents, which integrates: 1) a layer for building, measuring quality, and versioning RL environments; 2) an infrastructure layer for sandboxes, long-horizon rollout generation, checkpointing, and rollback; and 3) a post-training layer covering SFT, RL, and prompt optimization techniques like Japa (LLM-driven reflection for system prompt refinement). This integrated stack represents a converging vision for advanced agent development.

• Tags:
• Autonomous Agents
• Synthetic Data
• Data Curation
• Reinforcement Learning
• Enterprise AI
• Supervised Fine-Tuning
• RL Environments
• LLM Post-Training
• OpenThoughts
• Curator Tool
• Agent Reliability
• Reasoning Datasets