Waymo's Dmitri Dolgov: 20 Million Rides and the Road to Full Autonomy
Dmitri Dolgov, co-CEO of Waymo, discusses the 20-year journey from the DARPA challenge to full autonomy. He explains the Waymo Foundation Model—a multimodal world action model powering the driver, simulator, and critic—and how their "end-to-end plus" architecture enables superhuman safety and exponential scaling.
The Long Road to Autonomy: Persistence Through Hype Cycles
The journey to create a fully autonomous vehicle has been a nearly 20-year endeavor, originating from the spark of the DARPA Grand Challenge. For Dmitri Dolgov, this challenge was a “light switch moment” that combined fascinating technology with a powerful, real-world mission.
The Google self-driving car project began in 2009 with a small, intense team of about a dozen people. Their initial goals were to drive 100,000 miles in full autonomy and complete ten different 100-mile routes without human intervention—a feat that seemed impossible at the time but was accomplished in 18 months.
Waymo’s persistence through multiple AV hype cycles was grounded in a deep understanding of the problem’s complexity. Dolgov notes that technological breakthroughs often reshape the “early part of the curve,” making it easy to get started but not changing the long tail of challenges required to build a real product with superhuman performance. The team’s stamina came from two core beliefs:
- The Mission’s Importance: “Today worldwide somebody loses their life to a crash on our roads every 26 seconds.” This unacceptable status quo is a primary motivator.
- Realistic Expectations: Understanding that there are no “quick solutions or silver bullets” helped the team maintain the right mindset to go the distance.
The Waymo Foundation Model: A Multimodal World Action Model
At the heart of Waymo’s AI ecosystem is the Waymo Foundation Model. This single, powerful model underpins the three pillars of Waymo’s technology: the Driver (the agent in the car), the Simulator (for training and testing), and the Critic (for evaluation).
Dolgov describes it as a “multimodal world action language model” with several key capabilities:
- Multimodal: It reasons about the world using not just camera images and video, but also rich data from LiDARs and radars.
- World Action Model: It possesses a deep, precise understanding of 3D spatial properties, physics, and the behavioral aspects of other agents (cars, pedestrians, cyclists). Crucially, it’s not a passive observer but an active participant that must comprehend the effects of its actions on the world.
- Language-Aligned: It integrates the general world knowledge of a Vision-Language Model (VLM), providing a significant boost in understanding the semantics and deep social aspects of driving.
This model requires an extremely high degree of performance, accuracy, and realism in every aspect to function safely and effectively on public roads.
Beyond Vanilla End-to-End: The Power of Structured Representations
While the Waymo Foundation Model is an end-to-end system (from sensors to actions), Dolgov clarifies a “false dichotomy” in the current AI discourse. He argues the question isn’t “end-to-end or something else,” but rather “end-to-end, and then what else?” A basic, vanilla end-to-end system is insufficient for a fully autonomous product with superhuman safety that needs to operate at scale.
Waymo’s approach goes beyond the basic model by augmenting learned representations with structured, materialized intermediate representations. This “end-to-end plus” architecture provides critical advantages that are essential for a production-grade system:
- Runtime Validation: Enables extra safety checks and validation of the agent’s behavior in the physical world.
- Richer Training and Evaluation: Facilitates more sophisticated training recipes that are impractical in a pure end-to-end system, including closed-loop evaluation, closed-loop training, and more complex reward functions for reinforcement learning.
- Human Feedback Integration: Provides the necessary structure to incorporate feedback from human support staff and remote operators.
Hardware, Operations, and Exponential Scaling
Waymo has developed its hardware and software in tandem. The sixth-generation Waymo driver is the company’s most advanced sensor suite, with a focus on performance, simplification, drastic cost reduction, and readiness for high-volume production.
This technical maturity has unlocked a “phase transition” into exponential scaling:
- Ride Volume: It took 16 years to deliver the first 10 million fully autonomous rides, but only the last seven months to deliver the next 10 million.
- City Launches: After taking eight years to launch its service in four cities, Waymo recently launched in four new cities on a single day.
Launching in a new city involves operational setup, data collection, and extensive community engagement to earn trust. Today, the Waymo driver generalizes so well that new deployments are primarily a matter of rigorous, high-fidelity evaluation and validation.
Safety as the Non-Negotiable Foundation
From day one, safety has been the “non-negotiable foundation” of Waymo’s development process. This philosophy is embedded in the model architecture, training methodologies, and team culture, standing in contrast to the “move fast and break things” ethos.
The results are now statistically significant. Across over 170 million fully autonomous miles, the Waymo driver is more than 13 times safer than a human driver when it comes to collisions causing serious injury. At the current operational scale, this translates to preventing a serious injury every eight days.
This superhuman safety manifests in daily driving scenarios. Dolgov shared two mind-blowing examples:
- Emergency Maneuver: A Waymo vehicle demonstrated superhuman reaction time and precision by swerving and braking to avoid a person who unexpectedly fell from an electric scooter directly into its path.
- Seeing the Unseen: A Waymo vehicle, stopped at an intersection, detected a pedestrian hidden on the other side of a large city bus. It achieved this by bouncing its LiDAR signal under the bus and interpreting the sparse returns from the person’s feet. This allowed the AI to predict their trajectory and safely navigate around them as they emerged.