AI Is Missing This Data
David Fattal, CTO of Leia Inc., explains why stereo data is crucial for developing robust AI world models that can accurately understand 3D geometry and interact with the physical world. He contrasts it with monocular data, highlighting stereo's superior ability to ground AI in reality. Fattal introduces Leia's technology for glasses-free 3D displays on standard devices, predicting a "device-data flywheel" where accessible 3D hardware drives user-generated stereo content, providing essential data for AI advancement, a crucial differentiator for future AI companies.
David Fattal, founder and CTO of Leia Inc., discusses the critical role of stereo data in advancing AI, particularly for world models that aim to understand and interact with the physical environment. He posits that while monocular data (single viewpoint video) is abundant, it presents an “ill-posed problem” for accurately determining 3D geometry and spatial relationships. For instance, a single image cannot definitively distinguish a large, distant object from a small, close one.
Fattal highlights that human perception relies on binocular vision, leveraging two slightly different viewpoints to intuitively perceive depth and volume. He argues that the evolution of visual media, from black-and-white to color, motion, and touch, is driven by the desire to faithfully represent the real world, making stereo vision the logical next step.
For AI world models, which need to predict and plan actions based on environmental understanding, direct access to accurate geometry is paramount. While vast amounts of 2D video can pre-train models on textures, shading, and movement, stereo data grounds these models in reality, preventing “hallucinations” and providing a “truth about the geometry of the scene.” Fattal suggests that “one stereo image is worth maybe a thousand regular images” in its geometric information richness. While synthetic data from rendering engines (Unity, Unreal) can generate perfect multi-view data, it often lacks the “long tail” of real-world messiness and imperfections crucial for training robust, general-purpose AI. Similarly, LiDAR provides depth maps but has contextual limitations (e.g., roadside vs. indoor).
The current prevalence of monocular data in robotics and other AI applications is largely due to the scarcity of real-world stereo data. However, Fattal asserts that if readily available, such data would be adopted “in a heartbeat” by AI developers, as it provides geometry “for free,” reducing the effort required to infer spatial information. The challenge lies in creating a mechanism for widespread, crowdsourced stereo data capture, analogous to the current volume of 2D photos and videos.
Leia Inc. addresses this by developing technology to transform any screen into a spatial (3D) display, capable of projecting two distinct views to the user’s eyes without requiring cumbersome headsets. This experience, while not fully immersive like VR headsets, creates a “box” where content appears volumetric, allowing users to remain aware of their surroundings. Fattal emphasizes that many modern smartphones already possess dual cameras capable of capturing stereo data (“spatial option”), but the lack of a native 3D display on these devices limits immediate gratification and widespread adoption.
Leia envisions a “device-data flywheel”: as 3D-capable screens become ubiquitous and affordable (comparable to the cost of adding touch functionality), users will naturally capture and share 3D content. This surge in user-generated stereo data will, in turn, provide invaluable training data for AI models, leading to better experiences, and thus further driving demand for 3D devices.
Addressing the failure of 3D TV, Fattal points to key differences:
- Content: Previous 3D TV lacked content, and generation was manual and expensive. Today, AI can assist, and existing 3D game engines can easily render stereo views.
- Hardware: Early 3D TVs required glasses, which were cumbersome. Leia’s technology provides glasses-free 3D, seamlessly integrated into existing device form factors (phones, tablets, laptops) without degrading the 2D experience.
- Integration: The technology is an option that doesn’t force a particular consumption mode, making it more user-friendly and versatile.
Fattal predicts a near-term roadmap: within one year, a major OEM will release a high-volume 3D-enabled monitor or laptop; within two years, content platforms (e.g., Zoom, gaming) will naturally support stereo; and within three years, the first high-end mobile device with integrated 3D capture and display will emerge. He acknowledges that while current OEM focus is on AI, the strategic importance of new hardware for data collection (echoing rumors of OpenAI’s hardware ventures) will drive 3D adoption.
On the topic of self-driving cars, Fattal advocates for sensor fusion (cameras, LiDAR) for robustness, favoring redundancy in adverse conditions, though he notes the “Elon camp” argument that human-level vision (cameras only) is theoretically sufficient. Finally, he recommends the movie Minority Report as a cinematic vision for future spatial computing, where an “immersive space” is accessed across various devices, enabling seamless interaction and reinforcing the idea that “the future is already here, it’s just not evenly distributed.”