Building the "App Store" for Robots: Hugging Face's Thomas Wolf on Physical AI
Thomas Wolf, co-founder of Hugging Face, details the LeRobot project, aiming to replicate the success of Transformers in the robotics domain. He discusses the vision of creating a massive open-source community, tackling data scarcity, and the future of physical AI hardware, arguing that we are at a key inflection point for robotics similar to where LLMs were years ago.
Thomas Wolf, co-founder and Chief Science Officer of Hugging Face, believes we are at the same inflection point for robotics and physical AI today that we were for transformers and language models a few years ago. This realization, sparked by breakthroughs from academic labs showing robots performing complex tasks like folding clothes and cooking, led Hugging Face to start its robotics initiatives 18 months ago. The core insight was that hardware has been ready for some time; the missing piece was adaptable, dynamic software.
LeRobot: An Open-Source Ecosystem for Robotics
Hugging Face’s major initiative in this space is LeRobot, a project designed to replicate the success of the Transformers library for the robotics field. The goal is to create a central, accessible library that integrates the three core components of modern robotics:
- Policy and Models: The latest algorithms for training robots efficiently.
- Datasets: The data used to train these models.
- Hardware (Actuators): The physical components that perform actions.
By combining these elements, LeRobot aims to provide the fundamental building blocks for developers, startups, and researchers to build innovative applications on top. This follows the classic Hugging Face ethos: provide the platform and let the community create. The importance of this open, local-first approach is amplified in robotics, where safety is paramount. Running models locally is crucial, as a robot losing a Wi-Fi connection and malfunctioning is far more dangerous than an LLM hallucinating.
Building a Community of Roboticists
A primary goal for Hugging Face is to see if the massive community-building success in NLP can be replicated in robotics. The vision is to transition the millions of software developers who are now “AI-aware” into becoming roboticists. The early signs are promising, with the LeRobot community growing exponentially into the thousands, marked by events like a worldwide hackathon with 100 locations across six continents.
The community consists of three main personas:
- Traditional Roboticists: Experts in hardware who are eager to leverage modern AI to overcome the limitations of older software stacks.
- AI and Software Developers: Individuals intrigued by robotics as a physical manifestation of AI, who can now enter the field thanks to accessible software.
- Curious Non-Technical Users: People, including investors, who use affordable hardware like the $100 S100 robotic arm to get a physical, hands-on understanding of the technology’s potential.
The ultimate goal is to make robotics so accessible that even children could “vibe code” behaviors onto a robot.
Tackling the Data Bottleneck
Unlike language models, which can be trained on trillions of tokens from the internet, robotics suffers from a severe data scarcity problem. While one can use internet videos, training a robot for a specific task often requires new data collection. The primary challenge is diversity. A model trained extensively in one room will fail to generalize to an adjacent room with different lighting or wall colors.
Hugging Face’s strategy to solve this is two-fold:
- Community-Sourced Data: Incentivize the community to record and share datasets on the Hub. The aggregation of data from thousands of different environments could create a uniquely diverse “multi-location” dataset that enables robust generalization.
- Industry Collaboration: Encourage robotics companies to open-source their software and data. Since these companies primarily sell hardware, contributing to the open-source software ecosystem helps elevate the entire field, which in turn benefits their hardware sales.
The Future of Hardware: A Galaxy of Form Factors
Regarding hardware, Wolf is skeptical about the immediate dominance of humanoids. He points out two major issues:
- Cost: Humanoids require a large number of actuators, which constitute ~70% of a robot’s price. This makes it incredibly difficult to produce a humanoid for less than the price of a car (e.g., under $10,000).
- Social Adoption: The “uncanny valley” could present a significant barrier to widespread acceptance.
Instead of a single, expensive form factor, he envisions a “galaxy of different form factors” that are more accessible, affordable, and fun. This approach aligns with Hugging Face’s goal of democratization, preventing a future where robots are an elite technology only available to the wealthy.
Projects like the Reachy Mini, a $300 robot, are aimed at the consumer market for entertainment, education, and experimentation. The idea is to create an “App Store” for robot behaviors, where the community can endlessly create and share new functionalities, moving beyond the limited, pre-programmed behaviors of past consumer robots.
World Models and the Broader AI Ecosystem
World Models—controllable, photorealistic, and coherent video generation models—represent a potential breakthrough for robotics. They can be used to generate vast amounts of simulated data, a crucial component for training. This is particularly exciting because traditional physics-based simulation has not seen significant breakthroughs recently.
The conversation also touched on the open vs. closed model debate. The landscape is dynamic, with OpenAI returning to Hugging Face and Chinese companies surprisingly becoming champions of open source. This competition keeps the performance gap between open and closed models quite thin. Companies are increasingly turning to open-source models not just for cost savings, but for data privacy, control, and the freedom to innovate.
Open Science as the Foundation
Wolf’s passion for open source is rooted in his past as a physicist, where he struggled to access knowledge locked away in old Soviet-era papers. For him, open science is the next step. It’s not enough to release a model (“giving a fish”); it’s essential to explain how the model was trained (“teaching them to fish”). This empowers the community to build better models, creating a virtuous cycle. This philosophy is put into practice through detailed blog posts, books, and the creation of high-quality, open datasets like FineWeb, which details the process of data filtering and preparation.