Scale AI CEO on Meta’s $14B deal, scaling Uber Eats to $80B, & what frontier labs are building next

The general trend in AI is a significant shift from models that know things to models that can do things. As models become more knowledgeable, the critical question becomes, “What can it do for me?” This pushes the frontier toward agentic systems that can navigate complex environments and make decisions on a user’s behalf. This transition requires a fundamental evolution in how models are trained, moving beyond broad knowledge ingestion to acquiring specific, nuanced skills from human experts.

The Evolution of AI Training: From Generalists to Experts

The nature of data labeling and AI training has transformed dramatically. Eighteen months ago, a typical task involved simple preference ranking, such as determining which of two short stories was better. Today, the tasks are far more complex and require deep domain expertise.

• High-Skill Tasks: A single training task might now involve one of the world’s best web developers building an entire website to demonstrate a process, or a medical professional explaining a nuanced topic on cancer.
• Expert-Driven: These tasks can take hours and necessitate professionals with advanced degrees. At Scale AI, 80% of the expert network holds a bachelor’s degree or higher, and 15% have a PhD.
• Proactive Development: Scale AI has been at the forefront of this shift, often identifying model deficiencies in expert domains and proactively approaching model builders with solutions that leverage cadres of specialists.

Reinforcement Learning and The Challenge of Generalizability

To enable models to “do things,” they must be trained in simulated environments where they can learn to accomplish goals through trial and error—a process rooted in reinforcement learning (RL).

• RL Environments: These are effectively sandboxes, such as a configured Salesforce instance, where an AI agent learns to navigate the system, understand its data, and execute business processes.
• Training for Reliability: A critical part of this training is teaching the agent to recognize when the probability of success is low and to escalate to a human for feedback and guidance.
• The Generalization Problem: The number of possible environments, configurations, and user goals is virtually infinite. The central challenge is to collect training data that is generalizable across a broad spectrum of use cases. The goal is to provide model builders with the most valuable, generalizable data so they don’t have to train for every one of the “45 trillion combinations” of potential scenarios.

The Enduring Need for Human Judgment

The idea that AI will no longer require human data for improvement is far-fetched. Such a scenario would imply that no new human skill or knowledge is valuable enough to be incorporated into models.

• Digitizing Judgment: The current bottleneck is the digitization of true human judgment and subject-matter expertise. The meaning and importance of information are often context-dependent; a document’s interpretation can differ entirely between two companies based on their culture, objectives, and incentives.
• Enterprise-Specific Labeling: This need for contextual understanding is driving a trend where enterprises and government agencies are beginning to perform their own data labeling. Off-the-shelf models combined with Retrieval-Augmented Generation (RAG) can only go so far. To automate mission-critical workflows, models must be trained on the specific judgments of an organization’s internal experts.

Enterprise AI: Navigating the “Trough of Disillusionment”

While the hype around AI is immense, the on-the-ground reality of enterprise implementation is more complex. The high failure rate of GenAI pilots stems from the fact that AI is “easy to learn, hard to master.”

• The “Five Nines” Problem: Getting a proof-of-concept to 60-70% accuracy is relatively straightforward. However, achieving the reliability needed for a mission-critical process is an order-of-magnitude harder, akin to the difference between a single web server and a system with 99.999% uptime.
• Time and Rigor: Deploying a robust, automated process in an enterprise setting is not a matter of minutes; it takes 6 to 12 months to navigate legal, policy, regulatory, and change management hurdles to achieve a level of accuracy that everyone is comfortable with.

Lessons from Building Uber Eats

Building a new business, whether a startup or within a large company, requires a specific mindset and rigorous approach.

Independent Thinking and First-Principles Insight

To find true alpha in a crowded market, you must develop an independent perspective.

• Understand Underlying Incentives: Don’t take customer feedback at face value. Dig deeper to understand their core incentives, which may not always be purely financial but could relate to career growth or internal politics.
• Build Your Own Ground Truth: When launching Uber Eats, the team couldn’t get a clear picture of restaurant unit economics. Instead of relying on conflicting accounts, they bought sandwiches, weighed the ingredients (ham, cheese, bread), and constructed their own P&L model. This first-principles understanding gave them the conviction to propose a 30% take rate, knowing it aligned with the high incremental margin of a new delivery order for the restaurant.

Setting a High Bar for New Ventures

• Filter for Strong Business Models: Start by filtering ideas through the lens of proven, defensible business models: marketplaces, recurring revenue, network effects, and high switching costs. This quickly eliminates weaker ideas.
• Focus on Urgent Problems: You can build a product that provides significant value, but if it doesn’t solve an urgent, daily problem for the customer, you will face “a long road to a small town.” The most successful products address what the customer is thinking about day-to-day.

The Philosophy of Survival

In a tech culture that often glorifies risk, it’s crucial to remember that “not losing is a precursor to winning.”

• Survival is the Game: Most new ventures fail because the founders give up before they find the right product, customer, or market timing. You must survive long enough for the conditions for success to emerge.
• Calculated Risk: This doesn’t mean avoiding risk, but rather looking at the risk profile of every decision and consistently making asymmetrically positive bets that don’t compromise the long-term viability of the enterprise.

• Tags:
• AI Agents
• Reinforcement Learning
• Feature
• Data Labeling
• Enterprise AI
• Scale AI
• Uber Eats
• Jason Droege