The Moonshot Podcast Season 2, Episode 6: Silicon Horizons

The world is facing a compute crisis, where the demand for processing power, largely driven by artificial intelligence, is dramatically outpacing our ability to design and power computer chips. With the end of Moore’s Law, the 50-year-long self-fulfilling prophecy of doubling compute power every two years by shrinking transistors, new approaches are essential. This has led to a profound need for more efficient, specialized hardware.

If cars did what chips did, 50 years of Moore’s law, then you could drive to the sun in 1 hour on one gallon of gas.

Two moonshot projects from X, Project Positron and Project Bodger, tackled this challenge from different angles.

Project Positron: Building the Brains for Robots

Initiated around 2015, Project Positron, led by Tammo Spalink, aimed to build specialized hardware for the burgeoning field of AI, or as it was more commonly called then, machine learning. The team foresaw that robots and autonomous systems would require real-time decision-making capabilities at the edge, without relying on a connection to a data center.

This led to a focus on inference—the application of a pre-trained model—rather than the training process itself.

• Training vs. Inference: The distinction was analogized to a violinist’s practice versus performance.
  • Training is like the 10,000 hours of practice: it’s a long, computationally expensive process where latency isn’t the primary concern. The goal is throughput.
  • Inference is the live performance: it must be crisp, precise, and happen in real-time. The primary optimization target is response time.

Positron’s key insight was to design a chip architecture that fundamentally optimized for low-latency response time. This was a contrarian view at a time when the industry was still largely focused on throughput.

Technical Risks and Innovations

The Positron team made several bold, unconventional choices that were initially met with skepticism but have since become industry norms:

• Low-Precision Numeric Formats: They proposed using highly compressed 8-bit and even 4-bit integer representations for calculations. This dramatically increases speed and efficiency. At the time, the team was “actively laughed at” for this suggestion, as 16-bit precision was the standard. Today, low-precision formats are foundational to modern inference chips.
• Unconventional Software: The team wrote most of their simulation and design software in Haskell, a niche functional programming language, leveraging the team’s specific expertise.

Initially targeting Waymo as a primary customer due to the immediate need for instantaneous decision-making in self-driving cars, the project later pivoted. The team realized the immense volume of inference already happening in Google’s cloud for services like Translate and image recognition, which also benefit from low response times. The project eventually graduated from X and moved to the Google Brain team, the creators of the Tensor Processing Unit (TPU).

Project Bodger: Automating Chip Design with AI

While Positron focused on what a new chip architecture should do, Project Bodger, led by Raj Apte, addressed a more fundamental problem: how chips are designed. Raj observed that chip design was a bespoke, artisanal process, requiring large teams, years of effort, and budgets of $50-100 million per chip. This made exploration and iteration prohibitively expensive.

The moonshot goal for Bodger was to create a “hardware compiler”—a system that could automate the design process, turning it from a multi-year, multi-million dollar endeavor into something that could be done in hours for thousands of dollars.

Core Concepts and Techniques

Bodger’s approach relied on letting computers do what they do best: search vast, complex problem spaces without human biases or cognitive limitations.

• Inverse Design: Instead of a human designer making linear, step-by-step decisions, inverse design frames the problem differently. You define a figure of merit or a “goodness measure” that describes the desired performance characteristics (e.g., speed, power consumption). The computer is then tasked with searching the entire space of possible designs to find a solution that optimizes that metric.
• Breaking Human-Centric Abstractions: To manage complexity, human designers rely on simplifying rules and abstractions. One such rule is Manhattan geometry, where all conductive pathways on a chip are restricted to right angles, like streets on a city grid. This is easier for humans and early software to handle, but it’s not the most efficient path. Bodger removes these abstractions, allowing the AI to explore solutions with diagonal paths, curves, and other complex geometries that are physically optimal but would be incomprehensible for a human to design by hand.

We don’t really care if the chip is explainable. It just needs to be right. And if it’s right, that’s good enough.

By giving the AI this “unfair advantage”—a much larger and more complex search space—Project Bodger can generate novel, highly efficient chip layouts. The project uses a mix of techniques, sometimes “jumping” directly to a solution for smaller circuits and using more evolutionary, search-based methods for larger ones.

Like Positron, the Bodger team graduated from X to Google, where they now contribute to designing Google’s next-generation chips, applying their automated, AI-driven approach to tackle the immense compute demands of modern AI.

• Tags:
• Feature
• Inference
• TPU
• AI Hardware
• Chip Design
• Computer Chips
• Moore's Law
• Project Positron
• Project Bodger
• Inverse Design