How to Build the Future: Demis Hassabis
Demis Hassabis, CEO of Google DeepMind, outlines the remaining challenges on the path to AGI, including memory, continual learning, and true reasoning. He discusses how learnings from AlphaGo are shaping agent development, the strategic importance of powerful small models like Gemma, and his vision for AI as the ultimate tool for scientific discovery, offering a framework for identifying breakthrough opportunities and advice for founders building in the age of AI.
The current paradigm of large-scale pre-training and reinforcement learning from human feedback (RLHF) will likely be part of the final architecture for Artificial General Intelligence (AGI), but there are still fundamental pieces missing. There is a 50/50 chance that scaling existing techniques will be enough, but it is equally likely that one or two more big ideas are needed to bridge the gap.
The Path to AGI: Key Unsolved Problems
Several key areas remain unsolved and are critical for achieving AGI:
- Continual Learning and Memory: Current systems lack the ability to gracefully integrate new knowledge without catastrophic forgetting, a problem the human brain solves elegantly through mechanisms like memory consolidation during sleep. Today’s methods are a “bit unsatisfying,” often relying on brute-force approaches like expanding the context window. While a million-token context window seems large, it is insufficient for processing continuous, high-bandwidth data like live video over extended periods. Solving continual learning is a prerequisite for developing truly effective, adaptive agents.
- Reasoning and Introspection: Models exhibit a “jagged intelligence.” They can solve incredibly complex problems, like those at the International Mathematical Olympiad, yet fail at elementary reasoning tasks. This suggests a lack of introspection or self-monitoring of their own thought processes. When playing chess, for example, a model might identify a move as a blunder but play it anyway because it cannot find an alternative, highlighting a fundamental gap in its reasoning capabilities.
- Agents and Active Systems: The path to AGI runs through agents—active systems that can make plans and solve problems to achieve goals. We are only at the beginning of this journey. While current agents can be patched together to perform impressive tasks, they are not yet “fire and forget” systems. They lack the ability to learn and adapt to the specific context they are placed in, which is a key hurdle holding them back from more general-purpose utility.
The Spectrum of Models: From Frontier to Edge
There is a symbiotic relationship between massive frontier models and smaller, highly efficient ones.
- Distillation and Efficiency: A core strength at Google DeepMind is distilling the power of frontier models into much smaller, faster versions. There does not appear to be a theoretical information-density limit to this process yet. This capability is driven by the practical need to serve billions of users across Google products with low latency and cost.
- The Power of Small Models: The goal is for the capabilities of a frontier model to be available in tiny, edge-capable models within a year of its release. This is exemplified by the Gemma models. Smaller models are crucial for several reasons:
- Speed and Iteration: They allow for much faster collaboration, which can be more valuable than having a slightly more capable but slower model.
- On-Device Deployment: They enable powerful AI on devices like phones and robots, which is critical for privacy, security, and efficiency.
- Open Models: For edge models that are inherently vulnerable once deployed on a device, it makes strategic sense for them to be fully open, fostering a robust ecosystem.
AI for Science: The AlphaFold Blueprint
The ultimate motivation for building AI is to use it as a tool to accelerate scientific discovery. AlphaFold serves as the “prototypical example” of this vision, and its success reveals a pattern for future breakthroughs.
A scientific domain is ripe for an AlphaFold-style breakthrough if it meets three criteria:
- A Massive Combinatorial Search Space: The problem should be too vast for brute-force solutions, such as predicting protein structures or finding the optimal move in Go.
- A Clear Objective Function: There must be a well-defined goal to optimize for, like minimizing free energy for proteins or winning the game.
- Sufficient Data or a High-Fidelity Simulator: The system needs either a large dataset to learn from or an accurate simulator to generate synthetic data.
This pattern is applicable to numerous fields, including drug discovery, materials science, and mathematics. The next frontier is not just solving known problems but enabling AI to formulate novel, interesting hypotheses—to invent “Go” rather than just mastering it.
Advice for Founders
For founders building in the current landscape, especially in deep tech, it’s crucial to think long-term and strategically.
- Combine AI with Other Deep Tech: The most defensible and valuable opportunities often lie at the intersection of AI and another deep technology area, such as medicine or materials science, particularly if it involves the “world of atoms.”
- Tackle Hard, Meaningful Problems: Pursuing difficult, long-term goals is often no harder than working on superficial ones, just “differently difficult.” Given that life is short, it is worth investing your energy into something that will make a real difference.
- Plan for an AGI Inflection Point: If you are starting a 10-year deep tech journey today, you must account for the possibility that AGI will appear midway through. This means building something that can leverage AGI or will be a valuable tool for an AGI system. The future will likely involve general-purpose models using specialized tools like AlphaFold, rather than one monolithic “giant brain.”