Former Intel CEO: Why This is the Best Time to Build Hardware
Pat Gelsinger joins a16z's Raghu Raghuram and Guido Appenzeller to discuss how AI is reshaping chip design, why design speed alone will not fix semiconductor bottlenecks, and why power and energy capacity may ultimately limit AI growth. The conversation covers HBM and memory innovation, stacking limits, the shift from copper to optics, consolidation among AI chip startups, and what a VMware for AI agents might look like.
Opening: Pat Gelsinger on Chip Design, Bottlenecks, and the Road Ahead
The conversation between Raghu Raghuram, Guido Appenzeller, and Pat Gelsinger moves from Pat’s early Intel years through the coming shifts in chip design, memory, optics, power, and virtualization for AI agents. The thread running through all of it is that AI is accelerating design work, but the physical world (fabs, memory physics, power grids) is where the real constraints now sit.
From Tech School to the 386 and 486
Pat describes starting at Intel at 18, after skipping the last year and a half of high school for a tech school scholarship. He moved from technician to design team, became engineer number four on the 386, and later served as architect and design manager for the 486, all while pursuing his bachelor’s, master’s, and PhD. He recalls arguing with a Stanford professor over a carry look-ahead adder design, and working on built-in self-test with Ed McCluskey. His thesis advisor, John Hennessy, later became president of Stanford.
The 486 is framed as a turning point for EDA. Intel had to build its own hardware description language and a compiler for it, and there was no automatic place-and-route. The team worked with Alberto Sangiovanni-Vincentelli at Berkeley and his students on early automated placement, routing, and timing management. In Pat’s telling, the 486 helped usher in many foundations of the modern EDA industry.
How AI Changes Chip Design
Pat sees a parallel between the 486 era and today. AI tools can already handle much of the logic design, given enough transistor budget and a few experts guiding them. Analog design remains hard to automate because it requires large amounts of silicon data, so designers either get conservative or do that work the hard way.
The central point is that “whenever you have the technology to make something easy, the bottleneck moves somewhere else.” Pat walks through a concrete timeline: a team could design a chip in about three months with AI-assisted tools, but silicon processing takes around nine months, advanced 3D packaging adds roughly another three months, and then the part must be integrated into a rack-scale system. As he puts it, “nothing’s a chip anymore. It’s a rack.” By the time software is running on it, the workload understanding that motivated the design may already be outdated, which is how Graphcore’s design was overtaken by the market. He argues for new lithography approaches and design flows that do not require tens of millions of dollars in mask costs before prototyping, with the goal of compressing that cycle to a month or two.
He lists the next bottlenecks as memory bandwidth, power delivery, and thermal modeling. On HBM, his assessment is blunt: “HBM is a hideous memory. It’s just the best one that we have.” It has poor bit density, is limited by shoreline bandwidth, and suffers from thermal issues, since DRAM does not like to be hot and concentrating everything in one spot is a problem. On power, he notes that current simulation tools lack good 3D modeling for dissipation hotspots, and that designers effectively guard-band away a large share of the power envelope, possibly around 40 percent.
Will the AI Chip Explosion Consolidate?
The group examines the “100 AI inference accelerator chips” landscape and asks why investors fund so many. Pat argues the diversity is temporary and will converge, for three reasons.
First, the specialization is becoming too granular. Compute is splitting into prefill, decode, and now midfill tiers, and as reasoning models arrive, demand shifts back toward something more CPU-like. Pat does not believe fine-grained specialization at scale is sustainable, since workloads keep moving.
Second, the models themselves are approaching evolutionary breakthroughs. He suggests LLMs are reaching their limits for modeling 3D structures, molecules, chemicals, and imaging. He also expects that as AI systems optimize, they will pull some traditional HPC back in, including workloads needing 64-bit precision for chemical simulation.
Third, scale requires capital, workloads, and winners. Pat thinks it defies logic to expect a hundred chip companies to survive, and that large players such as OpenAI, Nvidia, and Anthropic will pick winners based on how hardware and software co-evolve.
Guido pushes back that a transition period naturally produces an explosion of approaches before they shake out, and that the real change is in software. Historically, instruction-set complexity forced consolidation to one or two architectures. AI agents can now write optimized kernels that humans could not, which lowers the software barrier to heterogeneity. Pat agrees the bottleneck shifts, since building and deploying these systems still takes time and enormous capital. He expects heterogeneity to be hidden underneath abstraction layers, as Nvidia’s acquisition-style absorption of Groq suggests, so that users will not know what is inside.
Memory Innovation After Thirty Years of Stagnation
Pat notes that, in the last 30 years, the industry has produced essentially zero major new memory architectures despite nearly a hundred attempts he is familiar with, including Intel’s Optane, which did not survive. Memory was historically a terrible business because of violent commoditization cycles: one profitable year followed by four losing ones. That has changed. All three major memory vendors now rank among the most valuable companies, and Pat jokingly notes that the memory industry has added roughly $2.5 trillion in market cap over the last four years.
He sees real opportunity in stacking and in breaking HBM’s shoreline bandwidth limit, mentioning Dmatrix and Cerebras-style techniques. He is also excited about new materials such as ferroelectrics, non-capacitive high-density memories, and stackable high-performance cells. He is skeptical of low-cost flash or MRAM approaches because of cell speed and performance limits. His conclusion: “memory innovation for the first time in 30 years is nigh upon us.”
How Tall Can Chips Get?
Stacking is constrained primarily by yield. The value of a stack grows with dimensionality, so per-layer yield must improve exponentially rather than linearly. A cracked die is a bad die regardless of built-in redundancy, and that is why Pat is skeptical of 16 or 32-high stacks. He expects the sweet spot to be three, four, or five layers. Even then, compute tiles are already reticle-limited, pushing designs toward chiplets with better yield and thermal properties. Adding power delivery in the Z dimension and redistribution layers to route signals makes the canonical package roughly eight layers thick, which Pat thinks is about the limit under today’s manufacturing.
On thermals, he argues that better power conversion reduces thermal chimneys, and that engineers are becoming plumbers, with liquid cooling and new materials such as diamond under consideration. He also weighs optics against stacking. Pat is wary of optical-electrical-optical conversions, which cost power and create losses. Energy per bit of communication is roughly a thousand times worse than energy per compute operation, so reaching large memory pools optically is expensive. He is similarly unconvinced by processing-in-memory approaches, which have been around for 25 to 30 years and impose constraints on workloads. He prefers high-bandwidth memory placed close to compute.
The Death of Copper and the Optical Transition
Pat says he declared the death of copper about 25 years ago and believes he will eventually be proven right. He sees all I/O eventually moving to optics, but he does not expect the core compute and memory complex to become fully optical. Copper has become a shorter and shorter waveguide, and at scale-up distances around five meters, copper is more expensive than optics at 100 meters. Physics is pushing the industry toward optics, but the supply chain has never been tested at scale. Pat expects 2028 and 2029 to be the inflection point for co-packaged optics (CPO) and similar approaches. He admits that building the NVL72 was “an engineering marvel and a manufacturing nightmare,” and that a mature optical supply chain would have produced better scale-up environments with better power and cost.
On network architecture, he references Nick McKeown’s observation that networks were built to handle any packet going anywhere, while AI traffic consists of large, predictable flows. That favors circuit or optical switching, and Pat thinks the boundary between scale-up and scale-out may dissolve as scale-up becomes large enough. Guido argues the distinction is largely artificial and reflects different protocols, front-end versus back-end, and that training and inference are converging, with continuous learning making the separation less meaningful over time. Pat agrees that training and inference may converge, while noting that inference still needs smaller node counts than training.
Energy as the Binding Constraint
The most urgent point is power. Pat argues that “energy capacity equals economic capacity” in the AI era. The US spent 10 to 15 years retiring coal faster than it added renewables, leaving national energy capacity roughly flat. Recent growth is only around 4 percent per year. Renewables carry deep dependencies on China, gas turbine lead times are now about eight years, and nuclear, which he calls a fabulous base-load technology, has not seen a new reactor come online in the US for about 20 years.
He predicts a headwind: “why build a new data center and buy the million GPUs if I can’t power them?” He expects more defaults on data center projects, and cites the early Oracle-linked data center financing trouble as the first indicator. Power delivery must also change. Moving to 800-volt DC architecture, which he jokes is “the revenge of Edison,” can eliminate conversion steps. Vertical GaN could convert 800 volts directly to 48, 12, or even 5 volts in a single step, and solid-state transformers and new cooling systems could revive interest in turbines and refrigerants. Pat sees this as a renaissance for hardware engineers after 25 years of relative quiet. He also points out that power per teraflop has been essentially flat across the last five Nvidia generations, which he considers unacceptable, and highlights superconducting approaches from Snowcap as a route to changing the physics.
A VMware for Agents
In the final section, the discussion returns to virtualization. Pat argues the virtual machine remains a foundational abstraction that deserves its place in the compute hierarchy. At VMware, the team managed the workload, network, and storage together. For agents, the same functions must be rebuilt: who manages agents, who sets security profiles, and who handles their performance.
A particularly interesting shift is that VMs are now being designed for agents rather than humans. This changes the form factor, marketing, and tolerated complexity, and it changes how quickly a system must start, since agents are far less patient than humans. Pat’s design constraint is clear: “how do I make agents great?” That means security, performance, abstraction, and migration, with something like vMotion for agents. He adds that humans still need to set policies, constitutions, and guardrails, and see dashboards, so both sides have to be designed. As he puts it, the hard design work is “abstracting the hardware and the operations for agents.”