The Godmother of AI on jobs, robots & why world models are next | Dr. Fei-Fei Li

Dr. Fei-Fei Li believes that while technology is a net positive for humanity, it is a double-edged sword. The future of AI is not predetermined; it is up to us to act as responsible individuals and communities to ensure it develops in a way that benefits humanity. This philosophy is captured in her statement: “There’s nothing artificial about AI. It’s inspired by people. It’s created by people. And most importantly, it impacts people.”

The Spark of the Deep Learning Revolution: ImageNet

The journey to modern AI was a long one, marked by a period known as the “AI winter” when public interest and funding were low. The field began its shift in the late 90s and early 2000s with the rise of machine learning, which married computer programming with statistical learning. This introduced the critical concept that machines must learn patterns from data rather than relying solely on rule-based programming.

Dr. Li’s work focused on visual intelligence, seeing it as a cornerstone of human cognition. The “north star” problem was object recognition, as humans perceive and interact with the world primarily at the object level. However, a singular pain point for all machine learning models at the time was the lack of training data.

This led to a crucial insight: human learning and evolution are “big data” processes. The overlooked ingredient for AI was a massive, well-structured dataset. This led to the creation of ImageNet, an ambitious project started in 2006 to map the entire world of objects with images.

• Scale: The project curated 15 million images across 22,000 object concepts, borrowing the taxonomy from the linguistic WordNet project.
• The Tipping Point: The 2012 ImageNet challenge marked the birth of the modern AI era. A team from the University of Toronto, led by Geoffrey Hinton, used the ImageNet dataset, a deep neural network (AlexNet), and two NVIDIA gaming GPUs to make a monumental leap in object recognition accuracy.
• The Golden Recipe: This moment established the three core ingredients of modern AI that are still in use today: large-scale data, neural network architectures, and powerful compute (GPUs). The technology behind ChatGPT, for instance, is a scaled-up version of this same fundamental recipe.

Beyond Language: The Next Frontier is Spatial Intelligence

While Large Language Models (LLMs) are incredibly powerful, they represent only one facet of intelligence. Dr. Li argues that much of human intelligence is built upon visual, perceptual, and spatial understanding. To move AI forward, we must go beyond language and embrace Spatial Intelligence.

Current models struggle with tasks that are simple for humans, like counting chairs in a video of an office or deriving the laws of physics from observational data, even with more data than Newton had. This indicates a need for new innovations beyond just scaling existing architectures. Spatial intelligence is the key to unlocking capabilities related to understanding, reasoning about, and interacting with the physical world.

World Models: Creating and Interacting with 3D Worlds

The practical implementation of spatial intelligence is through World Models. A world model is a foundational model that allows a user to:

• Create: Generate genuinely 3D, navigable, and immersive worlds from prompts (text or images).
• Interact: Move within the world, pick up objects, and make changes.
• Reason: Understand the spatial relationships and physics of the environment, enabling tasks like path planning for a robot.

This is the core focus of Dr. Li’s new company, World Labs.

Introducing Marble: The First Large World Model

Marble is the first product from World Labs, built on their frontier model. It is the world’s first publicly available generative model that can output genuinely 3D worlds from simple prompts—a concept they call “prompt-to-worlds.”

This differs fundamentally from video generation models. While a video model produces a passive, 2D projection of a world (like Plato’s allegory of the cave), a world model creates the underlying 3D structure. Users can explore this world from any angle, define their own camera paths, and export the assets for other applications.

Potential applications for Marble are already emerging:

• Virtual Production: Drastically reducing production time for movies and VFX by creating 3D environments for actors to perform in. A collaboration with Sony demonstrated a 40x speedup in production time.
• Robotics: Generating diverse, synthetic 3D environments for training and simulating robots, a major bottleneck in robotics research.
• Gaming and Design: Allowing developers and designers to rapidly create immersive worlds and assets.
• Scientific and Psychological Research: Creating controlled, immersive environments for experiments, such as studying how psychiatric patients respond to different scenes.

The Unique Challenges of Robotics

While world models are a critical missing piece for robotics, the field faces challenges that the “bitter lesson” of AI—that scale (more data, more compute) solves most problems—cannot address alone.

1. The Data Alignment Problem: LLMs benefit from a perfect alignment between training data (text) and output (text). Robotics requires action data in a 3D world, which is scarce. Web videos lack the necessary 3D action information, creating a mismatch between the available data and the desired output.
2. Robots are Physical Systems: Unlike LLMs, which are pure software, robots are physical systems that interact with the world. This makes them more analogous to self-driving cars. The journey from the first DARPA challenge-winning car to today’s Waymo vehicles has taken nearly 20 years, and the problem is still not solved. Robots are even more complex, as they must interact with and manipulate a 3D world, not just avoid objects on a 2D plane.

• Tags:
• Robotics
• World Models
• Spatial Intelligence
• Fei-Fei Li
• ImageNet
• Human-Centered AI
• Marble