The 100-person lab that became Anthropic and Google's secret weapon | Edwin Chen (Surge AI)
Edwin Chen, founder and CEO of Surge AI, discusses his contrarian, bootstrapped approach to building a billion-dollar company, the critical role of high-quality data and 'taste' in training advanced AI models, the pitfalls of current benchmarks, and why Reinforcement Learning environments are the next frontier in AI.
Edwin Chen, founder and CEO of Surge AI, shares a contrarian perspective on building a successful company in the AI space, emphasizing a deep focus on quality over the typical Silicon Valley playbook of blitzscaling and fundraising. Having bootstrapped Surge AI to over $1 billion in revenue with fewer than 100 employees, Chen argues for a future where small, elite teams can build fundamentally different and more innovative companies.
The Philosophy of High-Quality Data
The core of Surge AI’s success lies in a sophisticated understanding of data quality, which goes far beyond simple accuracy. Chen uses the analogy of writing a poem about the moon: a low-quality approach checks boxes like “is it a poem?” and “does it mention the moon?”. In contrast, a high-quality approach seeks “Nobel Prize-winning poetry,” evaluating for subtlety, imagery, emotional resonance, and uniqueness.
To achieve this, Surge has built a complex technological infrastructure to measure quality, gathering thousands of signals on its human data providers. This system is akin to Google’s search ranking, which not only removes spam but also identifies the best, most authoritative content. By analyzing everything from expertise and background to keyboard strokes and the ultimate impact of the data on a model’s performance, Surge can distinguish between someone who is good at writing poetry versus technical documentation, ensuring the right person is on the right task.
The “Taste” Factor in Differentiating AI Models
The long-standing superiority of models like Claude Code in specific domains like coding and writing is not just about having better data, but also about the “taste and sophistication” applied during the post-training process. AI labs make countless decisions that shape their models, including:
- Data Mix: The proportion of human data versus synthetic data.
- Optimization Goals: Prioritizing different qualities, such as visual design versus backend efficiency in code generation.
- Principled Stance: Deciding whether to optimize for flawed academic benchmarks for marketing purposes or to focus exclusively on real-world performance.
These choices are not purely scientific; they involve a degree of artistic judgment. The values and “taste” of the research team fundamentally shape the model’s final behavior and capabilities.
The Problem with Benchmarks and the Push Towards “AI Slop”
Chen is highly skeptical of current industry benchmarks and leaderboards, viewing them as dangerously misleading. He identifies two primary issues:
- Inaccuracy: Many benchmarks are themselves flawed, containing incorrect answers.
- Lack of Real-World Correlation: Benchmarks test objective, well-defined problems, which is different from the messy, ambiguous nature of real-world tasks. This explains why a model can win a gold medal in a math olympiad but still struggle to parse a PDF.
He is particularly critical of popular leaderboards like LM Arena, which he argues optimize for “AI slop.” Because users are casually “vibing” and picking the flashiest response, models are incentivized to use excessive emojis, bolding, and markdown, even if the content is hallucinated. This creates a negative incentive structure where labs chase leaderboard rankings for PR, potentially making their models worse on fundamental metrics like accuracy. This trend, combined with optimizing for engagement, teaches models to “chase dopamine instead of truth.”
The Future of AI Training: Reinforcement Learning Environments
The next frontier in AI training lies in Reinforcement Learning (RL) environments. These are rich, complex simulations of the real world—like a video game or a virtual startup complete with Slack threads, codebases, and Jira tickets.
In these environments, a model is given a complex, end-to-end task, such as figuring out why a service went down and fixing it. The model learns by taking actions, observing the consequences, and receiving rewards for making progress. This method exposes weaknesses that isolated, single-step benchmarks cannot. It tests a model’s ability to reason, plan, and act over long time horizons in a messy, dynamic world.
Critically, it’s not just about the final outcome but the entire trajectory. A model might randomly stumble upon the correct answer after 50 failed attempts. By analyzing the entire path, trainers can teach the model to be efficient, reflective, and correct in its process, not just its final output.
The Evolution of Post-Training and Model Differentiation
AI training has evolved through several stages:
- Supervised Fine-Tuning (SFT): Analogous to mimicking a master.
- Reinforcement Learning from Human Feedback (RHF): Learning by having a human compare and rank different outputs.
- Rubrics and Verifiers: Learning by being graded against detailed criteria.
- RL Environments: Learning through hands-on practice in a simulated world.
Looking ahead, Chen predicts that AI models will become increasingly differentiated, not just in capability but in personality and values. This will be a direct reflection of the objective functions their creators choose. For example, one AI might be designed to maximize user engagement by encouraging endless iterations on an email draft. Another, built on different values, might optimize for user productivity by stating the email is good enough and encouraging the user to move on. The “perfect” model behavior is subjective, and this will lead to a diverse ecosystem of AIs shaped by the distinct principles of the companies that build them.