How End-to-End Learning Created Autonomous Driving 2.0: Wayve CEO Alex Kendall

Alex Kendall, CEO of Wayve, discusses the architectural shift from AV 1.0's hand-engineered robotics to AV 2.0's end-to-end deep learning. He explains how Wayve's generalization-first approach, powered by world models and diverse data, allows them to scale across hundreds of cities and multiple automotive OEMs, creating a path toward a general-purpose embodied AI foundation model.

4 min read
How End-to-End Learning Created Autonomous Driving 2.0: Wayve CEO Alex Kendall
source: https://www.youtube.com/watch?v=8x_O8BeGNTw

Wayve was founded on the contrarian belief that the future of autonomous vehicles (AV) would not be a hand-engineered system reliant on infrastructure like high-definition maps, but rather an intelligent machine built on end-to-end deep learning. This approach, termed AV 2.0, replaces the classical robotics stack—where perception, planning, and control are separate, hand-coded components—with a single, large neural network. While this was a radical idea in 2017, the breakthroughs in AI, particularly with large language models, have shifted the industry’s mindset, validating that a data-driven, learned solution can outperform brittle, rules-based systems.

The AV 2.0 Paradigm: Generalization Over Hand-Engineering

The core challenge in autonomous driving is generalization: the ability to reason about and safely navigate situations never encountered before. The traditional AV 1.0 approach, with its reliance on HD maps and extensive hand-coded rules for edge cases, struggles to scale beyond geographically-fenced areas.

Wayve’s AV 2.0 architecture is designed for generalization from the ground up. This allows for rapid adaptation and deployment:

  • New Vehicles: Wayve adapted its system to a new Nissan vehicle in Tokyo in just four months.
  • New Geographies: After learning to drive on the right side of the road in the US, the system required exponentially less data to adapt to Germany, demonstrating its ability to transfer knowledge. The system now operates in over 500 cities.

The ultimate ambition is to build an “embodied AI foundation model”—a single, general-purpose intelligence that can be quickly adapted to different vehicles, sensor suites, and applications for various fleets and manufacturers, amortizing the cost of intelligence across the entire ecosystem.

Reasoning Through World Models

A key enabler of this generalization is the use of world models. These are generative models that learn to simulate how the world works, allowing the AI to reason about potential future scenarios.

  • Function: A world model acts as an internal simulator, enabling the system to predict how other agents will behave and understand the consequences of its own actions.
  • Emergent Behavior: This deep understanding leads to sophisticated, human-like emergent behaviors. For example, the vehicle will autonomously nudge forward at an occluded turn to get a better view or slow down in foggy conditions, adapting its driving to its level of perceptual certainty. This is crucial for navigating complex, multi-agent scenarios safely and smoothly.
  • Data Efficiency: While not a replacement for real-world data, world models magnify and recombine existing data, creating novel synthetic scenarios that improve learning efficiency and robustness.

Data Strategy and Go-to-Market

Wayve’s data strategy prioritizes diversity over sheer volume. By partnering with various fleets, dash-cam providers, and manufacturers, they aggregate data from a wide range of vehicles, sensor configurations, and geographic locations. They employ unsupervised learning techniques to mine this data for unusual and challenging scenarios, creating a targeted curriculum to continuously improve the model.

The go-to-market strategy is centered on partnering with the world’s largest automotive OEMs. This provides a direct path to scale far beyond city-by-city robo-taxi deployments.

  • Software-Defined Vehicles: Manufacturers are now producing vehicles with the necessary infrastructure (onboard GPUs, surround sensors) to run advanced AI.
  • Path to Scale: With 90 million cars produced annually, partnering with OEMs allows Wayve to deploy its AI as a native software integration, avoiding costly hardware retrofits and enabling mass-market adoption.
  • Collaboration: Wayve learns from the auto industry’s expertise in reliability, quality, and efficiency, while providing the core AI to power everything from driver-assist features to fully autonomous “eyes-off” systems.

Technical Architecture: Sensors, Language, and Personalization

  • Sensor Agnosticism: Instead of debating camera-only versus LiDAR, Wayve’s AI is trained to be sensor-agnostic. It can operate on camera-only systems but also seamlessly incorporates radar and LiDAR data when available. The broader industry is coalescing around a scalable architecture of surround cameras, surround radar, and a front-facing LiDAR, which provides the redundancy to achieve superhuman performance.
  • Language Integration: In 2021, Wayve developed “Lingo,” a Vision-Language-Action model. Integrating language provides three key benefits:
    1. Better Representations: Pre-training with language improves the model’s foundational understanding.
    2. Product Features: It enables a conversational interface, allowing the user to interact with the vehicle like a human chauffeur (e.g., “drive more assertively”).
    3. Introspection: Engineers and regulators can query the model in natural language to understand its reasoning and diagnose behavior.
  • Foundation Model Approach: A single base model is trained on all data. This model is then efficiently fine-tuned and optimized for a specific OEM’s vehicle, sensor configuration, and desired driving personality, allowing for both scale and customization.

The Future: AV 3.0 and Embodied AI

The current AV 2.0 paradigm is about embedding intelligence within the vehicle. The next frontier, AV 3.0, could involve taking intelligence outside the vehicle. This might look like a mesh network where autonomous vehicles communicate with each other to coordinate traffic flow, potentially eliminating the need for traffic lights.

Ultimately, the technology being developed is a general-purpose navigation agent. This foundation model for mobility has applications far beyond consumer cars, extending to trucking, logistics, and eventually other robotics domains like manipulation and humanoid robots. The path to physical AGI will require further breakthroughs in measurement, algorithmic capabilities, and the efficiency of the engineering infrastructure used to train these massive models.

Share post

Stay In The Loop! Subscribe to Our Newsletter.

Get updates straight to your inbox. No spam, just useful content.

Recommended for You

Fully autonomous robots are much closer than you think – Sergey Levine

Fully autonomous robots are much closer than you think – Sergey Levine

Sergey Levine, co-founder of Physical Intelligence, outlines the path to general-purpose robots, predicting a 'self-improvement flywheel' could lead to fully autonomous household robots by 2030. He discusses the architecture of vision-language-action models, the critical role of embodiment in solving the data problem, and how robotics will scale faster than self-driving cars.

Waymo's EMMA: Teaching Cars to Think - Jyh Jing Hwang, Waymo

Waymo's EMMA: Teaching Cars to Think - Jyh Jing Hwang, Waymo

An exploration of Waymo's research into EMMA, an End-to-End Multimodal Model for Autonomous Driving. This summary details how foundation models like Gemini are being adapted to create a single, generalizable system that processes raw sensor data directly into driving decisions, aiming to solve the long-tail problem and improve scalability. It also covers the use of generative AI for advanced sensor simulation and model evaluation.