Chelsea Finn: This is the State of the Art in Robotics

Chelsea Finn, co-founder of Physical Intelligence, discusses the evolution of robotics towards general-purpose systems capable of robust operation in the real world. The core challenge lies in making robots reliably autonomous for extended periods, moving beyond human-babysitted applications.

The Need for Real-World General-Purpose Robots

Finn highlights that while AI has seen success in areas like recommendations and ad ranking where human oversight mitigates errors, physical AI and robotics demand far fewer mistakes due to their direct interaction with the physical world. She draws a parallel to Waymo’s success in autonomous rides, demonstrating the feasibility of reliable, autonomous systems.

Achieving Reliability Through Reinforcement Learning

To achieve high reliability, conventional machine learning’s iterative process of data collection, model training, and evaluation is insufficient. Instead, an automated self-improvement loop is necessary. Physical Intelligence employs a scalable reinforcement learning (RL) recipe for robotics, which contrasts with language model RL due to the high cost of physical interaction:

• Efficient Iteration: Unlike language models that can afford millions of attempts in data centers, physical robots incur significant hardware and real-world interaction costs. A million one-minute robot trajectories would equate to 700 robot days.
• Human Intervention for Efficiency: To avoid costly “dead-end trajectories” (e.g., a robot trying to fold two flush boxes), human teleoperation intervenes to show the robot how to recover, making data collection more efficient.
• General Purpose Value Functions: Instead of estimating value for individual prompts, a general-purpose value function is trained on diverse robot experiences. This function learns what constitutes progress or failure across different tasks (e.g., unfolding a shirt vs. retrieving an item from a fridge), significantly reducing the attempts needed for improvement.

This RL framework has led to significant improvements, including a 2x increase in throughput and achieving over 90% success rates on complex tasks like making espresso. The same approach has been applied to diverse real-world workflows, such as constructing and stacking boxes for a chocolate factory and folding laundry in unseen environments.

The Critical Role of Memory for Long-Term Autonomy

Most state-of-the-art robot foundation models lack memory, operating only on current sensor observations. However, memory is crucial for long-horizon, non-repetitive tasks that involve multiple sequential steps. Naively incorporating memory (e.g., 10 seconds of video from multiple cameras at 50Hz) can result in prohibitively large token counts (e.g., half a million tokens).

Physical Intelligence developed a multi-scale memory solution:

• Short-term Video Memory: Efficiently processed video memory for roughly 10 seconds.
• Long-term Textual Memory: For spans of minutes or hours, memory is summarized in compressed text, tracking completed steps and overall progress. This allows robots to perform complex, multi-step tasks like cleaning an entire kitchen autonomously for 10-15 minutes.

Building General-Purpose Models: The GPT Era of Robotics

Finn charts the evolution of generalist AI (from specialized models to fine-tuned pre-trained models to out-of-the-box generalists like GPT-2/ChatGPT). Robotics has historically lagged, often requiring bespoke datasets and training from scratch. Physical Intelligence aims to propel robotics into the “GPT era” with two key goals:

1. Out-of-the-Box Models: Moving from fine-tuning for every task to a single general-purpose model that works without task-specific training. Current best robot performance almost always requires fine-tuning.
2. Compositional Generalization: Inspired by DALL-E, enabling models to combine concepts not explicitly seen together in the training data, demonstrating conceptual understanding and data efficiency.

Their recipe for achieving these goals involves:

• Vast, Diverse Data: Combining diverse robot demonstration data (including low-quality), policy rollout data, videos of humans, and web data.
• Sufficient Model Capacity and Detailed Prompting: Training large models and prompting them with extensive context: memory, general instruction, subtask instruction, metadata (data quality, episode length), and optional subgoal images.

Their PIO7 model is a single model capable of diverse tasks like folding shirts, precise assembly, and replacing trash bags. Quantitatively, PIO7 matches or outperforms fine-tuned specialist models (PIO6) across tasks, demonstrating strong out-of-the-box performance.

Demonstrating Compositional Generalization

PIO7 shows compositional generalization in several ways:

• Novel Object Interaction: Interacting with a rare appliance (air fryer) not significantly represented in training data, combining skills like opening/closing with the new object.
• Platform Transfer: Transferring skills (e.g., folding clothes) from one robot platform to a significantly different one (e.g., a large industrial arm) without any platform-specific training data. This demonstrates a learned equivariance, allowing the model to adapt behaviors across different embodiments.

Quantitative ablation studies confirm the importance of diverse data and detailed prompting; the latter even allows the model to extract value from low-quality data.

The Future: Robotics in the GPT Era

Finn concludes that physical intelligence is now firmly in the “GPT and DALL-E like era” for robotics. Their models are being deployed in real-world applications by YC companies (e.g., laundry folding, warehouse packaging) and are adaptable to diverse robot embodiments (bimanual platforms, drones, surgical robots, tractors).

Q&A Highlights:

• ChatGPT Moment for Robotics: The “moment” might not be as explosive as ChatGPT due to slower physical deployment channels, but the capabilities are rapidly approaching.
• Generalist Policy Adoption: Small teams should start with generalist policies and fine-tune them, often leveraging open-source models (like PIO0 and PIO5). Specialists are only preferable in extremely constrained environments.
• PhD vs. Industry: A PhD offers invaluable experience in handling uncertainty and problem selection, essential for frontier AI. Industry provides diverse opportunities in engineering (software, hardware, ML infrastructure) and research.
• Robotics Internet-Scale Data: The equivalent to language data is robot operating experience (teleoperated and autonomous attempts). Human videos and web data are useful supplements, but direct robot experience on its own platform is critical for effective learning.
• Open-Source vs. Proprietary: While the cost of embodied data and hardware makes open-sourcing challenging, Finn is optimistic about a strong open-source community, noting that even closed-source companies contribute to open source to build ecosystems.
• Model Output: Models typically output target joint positions or gripper positions, which are then handled by a PD controller. This level of control works well and is not a current bottleneck.
• Imagination: The PIO7 model can imagine future states, which empirically helps performance, though the model performs surprisingly well even without it.
• Speed Improvement: Speed is a key area of focus; reinforcement learning and specific techniques like “RL token” have shown super-human speed. The challenge lies in making data faster or policies faster than the data.
• Surprising Task: Observing the robot learn an emergent equivariance between its left and right hands, enabling it to transfer actions without explicit training data.
• Breaking into Robotics: Join a robotics company as a software engineer, or, as exemplified by a Physical Intelligence employee, engage in personal projects with cheap robots and open-source models, sharing the results to demonstrate passion and capability.

Physical Intelligence is actively hiring for those passionate about these advancements.

• Tags:
• Foundation Models
• Robotics
• Reinforcement Learning
• Feature
• Physical Intelligence
• Robot Learning
• AI Deployment
• General-Purpose AI
• Long-Term Autonomy
• Compositional Generalization
• Human-Robot Interaction