The Startup Powering The Data Behind AGI

The Problem with Traditional Data Labeling

Edwin Chen’s journey to founding Surge AI began with a common pain point he experienced as an ML engineer at companies like Twitter. A simple task like building a sentiment classifier required 10,000 labeled tweets, but the internal system consisted of two people hired from Craigslist working through a spreadsheet. The process was slow, and the quality was poor, as the labelers didn’t understand nuances like slang or hashtags.

This frustration highlighted a deeper issue. When trying to move beyond simple engagement metrics like clicks and retweets—which create negative feedback loops rewarding clickbait and low-quality content—there was no infrastructure to generate the complex, principle-aligned data needed to train more sophisticated recommendation algorithms. This gap in the ecosystem, a recurring problem at major tech companies, inspired the creation of Surge in 2020.

A New Philosophy: From Commodity to Complexity

Surge was founded on the principle of moving away from “commodity labeling.” Past data solutions were designed for low-skill tasks with a low complexity ceiling, like drawing a bounding box around a car. In this paradigm, the quality difference between an average person and a brilliant mind like Terence Tao is negligible.

However, modern AI ambitions, such as writing evocative poetry or solving relativistic equations, require feeding models with a virtually unlimited amount of intelligence. Surge differentiated itself by focusing on these high-complexity, high-expertise tasks from the beginning, building a technology-first platform to manage quality at scale. Chen views Surge as a “research company,” but one that researches data instead of algorithms.

Rethinking Quality: Beyond Checkboxes and Inter-Annotator Agreement

A core tenet of Surge’s approach is a fundamental shift in how quality is measured, moving beyond the “old school” computer vision mindset.

• The Fallacy of Checklists: When training a model to write a poem, a simple checklist (e.g., “is it 8 lines?”, “does it mention the moon?”) results in terrible, uninspired output. Similarly, simply hiring people with PhDs in English Literature is not the answer, as academic credentials do not guarantee creative talent. The goal isn’t to create content that checks boxes, but content that a “Nobel Prize laureate would write.”

• The Limits of Inter-Annotator Agreement (IAA): In the old world of simple classification (cat vs. dog), high IAA is a useful signal of quality. However, for creative and complex tasks, it’s a flawed metric. There isn’t one correct way to write a great poem or prove the Pythagorean theorem; there are thousands of valid, creative approaches. Optimizing for agreement forces a regression to the mean, producing “the lowest common denominator of things that people want, which actually turns out to be kind of trashy and low quality.” This approach stifles the very creativity and intelligence that models need to learn.

The Dangers of Benchmark Hacking

Chen offers a strong critique of the industry’s reliance on certain benchmarks, particularly LMSYS, which he argues has “set the industry back by at least a year.”

The problem stems from the evaluation methodology: anonymous users, often high schoolers, rapidly vote on paired model outputs. They don’t have the time or expertise to verify facts or judge quality deeply. Instead, they are swayed by superficial characteristics. As a result, the easiest way to climb the leaderboard is to make models produce longer responses with more emojis and bold formatting.

This creates perverse incentives for researchers, who are told their promotions depend on leaderboard rankings. Instead of improving fundamental capabilities like coding or reducing hallucinations, they are pushed to optimize for these superficial traits, sometimes even making their models worse on core competencies to appear more compelling to casual evaluators.

The Evolution and Future of Data Collection

The nature of data required for frontier models is constantly evolving.

• From SFT to RLHF: Surge holds a strong conviction that Reinforcement Learning with Human Feedback (RLHF) is far more effective than Supervised Fine-Tuning (SFT). While SFT is useful for bootstrapping, RLHF is more efficient and teaches the model latent preferences about what is both good and bad. Chen points to the Llama 2 paper as a public validation of this approach, which Surge had been advocating to its customers.

• Current Trends:

  • Multimodal Complexity: Moving beyond text to models that understand images, audio, and video simultaneously. For example, asking a model to create a program that simulates the physics in a video captured on a phone.
  • Hyper-Specialization: Models still struggle with cultural and linguistic nuance. This necessitates generating highly specific data, such as “coding in Argentinian Spanish” or “legal and financial expertise in Bolivia.”
  • Depth of Expertise: Tasks have shifted from taking seconds to complete to requiring days or even weeks of deep, focused work by experts to generate a single data point.

• Future Data Needs:

  • Scientific Discovery: Progressing beyond graduate-level STEM problems to data that enables models to act as true “scientific collaborators,” helping researchers generate and test new hypotheses.
  • Long-Horizon Tasks: Training agents on tasks that take days or weeks to accomplish.
  • Agentic Reasoning in RL Environments: Building rich, simulated “video game-like universes” (e.g., a simulated AI startup with its own Gmail, Slack, and codebases) to stress-test agentic planning and reflection, especially in messy, unpredictable scenarios like a sudden AWS outage.
  • Personalization: Moving beyond surface-level personalization to capture the deep, latent preferences and “taste” that make interactions feel unique to an individual.

On Synthetic Data

While synthetic data has its uses, Chen believes its capabilities are often overestimated. It tends to make models proficient at synthetic, academic-style problems but can make them significantly worse at real-world tasks. He shared an anecdote of a model’s performance tanking across the board after being trained on millions of synthetic math problems. In many cases, a few thousand examples of high-quality human data prove more valuable than millions of synthetic ones, and a significant portion of Surge’s work involves “cleaning up” the negative effects of poorly implemented synthetic data strategies.

• Tags:
• AI Agents
• Synthetic Data
• Benchmarks
• Data Labeling
• RLHF
• Data Quality
• LLM Training
• Surge AI