Agents are Robots Too: What Self-Driving Taught Me About Building Agents — Jesse Hu, Abundant
Drawing surprising parallels between AI agents and robotics, this talk argues that the agent development community is repeating a key mistake from the self-driving industry: underestimating the difficulty of action and over-focusing on reasoning. It covers essential robotics concepts like DAgger, MDPs, simulation, and the critical importance of a robust offline infrastructure, explaining why perfect reasoning doesn't guarantee successful execution in the real world.
The central lesson from the self-driving industry is that everyone initially believed perception was the hard problem and planning was easy. It took nearly a decade to learn the reverse was true. Today, the field of AI agents is making the same mistake. We see models with sophisticated reasoning capabilities that can generate flawless plans, yet these agents consistently fail when executing those plans in the real world. Predictive models are not action models, and perfect reasoning does not guarantee successful execution.
The 1% Model and the 99% Infrastructure
Just as in robotics, the model itself is only 1% of the total work. The other 99% is the vast infrastructure that supports it, often called the offline stack.
- Robotics: This stack includes hardware, sensors, actuators, and the entire offline system for simulation, training, and evaluation.
- Agents: An agent’s “body” is its set of tools, APIs, terminal, browser, or even a full Virtual Machine (VM). The supporting offline stack is just as critical, encompassing continuous retraining, monitoring, and human feedback loops. The team with the best offline stack often wins because it enables faster and more reliable development.
Core Robotics Concepts for AI Agents
Several foundational concepts from robotics directly apply to building robust agents.
From Open-Loop to Closed-Loop Control
An open-loop system takes an action without observing the outcome. For example, running a bash command without checking if it succeeded or what it outputted. A closed-loop system incorporates feedback to adjust its behavior. If a robot turns its wheel, it measures how much the car actually turned to recalibrate. Agents need to move towards closed-loop systems to handle failures, observe long-running processes, and react to unexpected outcomes.
Stateful vs. Stateless Environments
We are moving from stateless agents that operate in isolated sessions to stateful agents that persist in environments like VMs. A stateful agent must contend with the initial state of the world (e.g., existing files, running processes) and its actions have lasting consequences. This dramatically increases complexity for both online execution and offline evaluation.
The Problem with Imitation Learning (DAgger)
Training agents on human demonstrations (Imitation Learning) is effective until the agent encounters a novel situation not present in its training data—an out-of-distribution state. This is a well-studied problem in robotics (known as DAgger). Once an agent deviates slightly from the demonstrated path, it can enter a cascade of errors from which it cannot recover. This is frequently seen in browser agents that get confused by an unexpected pop-up.
Actions have consequences. We’re not just dealing with prediction models. We’re dealing with a whole new paradigm in which you predict, you act, and then you deal with the consequences of that action and then re-evaluate everything you’ve done before.
Input and Action Spaces
Current agents often operate with a discretized sense of time, taking turns in a conversational manner. This is an implicit design choice that prevents real-time interaction. In robotics, one explicitly decides how frequently to sample the environment (e.g., 50Hz). We must also think more creatively about action spaces. The Terminus agent, for instance, interacts with a TTY stream at the character level, allowing for fine-grained control like Ctrl+C, moving beyond coarse tool calls.
The Hill Climbing Development Loop
Developing complex agents is a “hill climbing” process of iterative experimentation. A sophisticated approach, modeled after self-driving development, involves a virtuous cycle:
- Learn & Simulate: Train a model and then run it through extensive simulation. Simulation allows you to test against countless scenarios and “counterfactuals” (what-if scenarios) with confidence before real-world deployment.
- Deploy & Log: Once deployed, the agent’s real-world interactions are logged.
- Feedback Loop: These logs are invaluable. They are fed back into the simulation engine to make it more realistic and are used to triage specific failures. Analyzing individual failures provides far more actionable insight than a single benchmark score.
Towards a Formal Practice of “Agentics”
To advance from hacking on demos to building reliable systems, the field must formalize its practices by adopting these core concepts from robotics. This new discipline can be thought of as “Agentics”. By understanding principles like closed-loop control, MDPs, the challenges of statefulness, and the necessity of simulation, we can build agents that are truly effective in the real world.