Why AI Demand Is Outrunning Compute Supply
A deep dive into the current AI boom, exploring why demand for intelligence is likely underestimated and the outcome won't necessarily be winner-take-all. The discussion covers the rapid economics of AI infrastructure, the risk of an AI shortage over a bubble, the potential for orbital compute and re-industrializing America, and NVIDIA's central role as the "central bank of AI" in a multi-model future.
Gavin Baker and David George discuss the state of the AI boom, challenging conventional bearish sentiment and proposing an “everyone wins” scenario where frontier labs, open-source models, applications, clouds, and NVIDIA all thrive. They delve into the economics of the infrastructure buildout, the risk of an AI shortage, and the transformative potential of AI.
Challenging the Bear Case: An “And” Not an “Or” Scenario
Despite current public market downturns for some AI names, Gavin Baker struggles to find quantitative data points suggesting a slowdown in the AI space. He notes acceleration in OpenAI, open-source, and Grok (especially after Grokbot). The discussion suggests an “and” scenario where many players succeed simultaneously, including Anthropic, OpenAI, SpaceX, Meta, Google (selling TPUs), open-source, neo-clouds, inference clouds, and applications, with NVIDIA at the center.
Frontier Labs and the Training-Inference Trade-off
Frontier model companies like Anthropic and OpenAI are expected to reinvest all incremental profits into training, not prioritizing free cash flow. This reflects a deep belief in scaling laws. The trade-off between allocating compute to training versus inference can dramatically impact a lab’s reported revenue, making their financial performance volatile and unpredictable for public markets. Microsoft’s strategy is seen as more aligned with a future of multi-model architectures, where an ensemble of models is used, rather than a single dominant model.
The Economics of AI Infrastructure: Rapid Payback and Underestimated Demand
Compute investments currently offer unusually fast payback periods, with examples like Nebius and CoreWeave seeing 9-10 month paybacks. SpaceX clusters offer even faster returns. These investments are highly financeable, with firms like Blackstone and KKR providing low-cost capital, driven by extended useful lives of GPUs and increasing monetization per gigawatt. This suggests that the true equity payback could be well within a year.
The demand side for AI is seen as massively underestimated. Currently, a small number of heavy users (sub-10 million engineers) drive significant token spend, but this is a tiny fraction of the 1.5 billion knowledge workers. The potential for diffusion is enormous; Gavin’s firm saw a 100x increase in internal token consumption in five months. The shift from reactive coding (e.g., Claude Code) to fully autonomous agents (e.g., Grokbot) is a significant step, leading to potentially endless token consumption.
AI Shortage vs. Bubble: Reindustrializing America
The hosts debate whether the bigger risk is an AI bubble or a massive supply shortage. Gavin argues that the world is heading towards an under-supply of compute through 2028, exacerbated by political dynamics and resistance to data center construction. This could lead to “compute inequality” and higher token prices. He highlights the positive societal impact of data centers, which are re-industrializing America by creating jobs, boosting tax revenue in small towns, and improving environmental practices. He criticizes the abstract nature of current AI advocacy and calls for the industry to communicate tangible, everyday benefits to combat negative narratives, some of which he suspects are foreign-funded.
The Future of Compute: Orbital Data Centers and SpaceX’s Vision
Orbital data centers, while sounding futuristic, are presented as a solved engineering problem by SpaceX. The economics become compelling with Starship reusability, driving launch costs down to under a billion dollars, while terrestrial power and cooling costs are inflationary. Elon Musk and Jensen Huang have reportedly co-designed a “Ruben rack” for launch in late 2027. SpaceX’s broader ventures, like Starlink (a $2 trillion market opportunity addressing unserved areas and offering massive consumer surplus), and even asteroid mining (Psyche asteroid containing more precious metals than Earth’s crust), illustrate a vision of humanity becoming multi-planetary and moving heavy industry off-Earth, as envisioned by Jeff Bezos.
Multi-Model Architectures and the Abstraction Layer of Intelligence
The future of AI applications is likely to be an “ensemble of models” – a hybrid approach combining highly capable open-source models (potentially funded and led by chip companies like NVIDIA) with fine-tuning on proprietary enterprise data, alongside frontier models for complex planning tasks. This allows companies to own and control their intelligence. Microsoft is seen as well-positioned for this future. The question of “who becomes the abstraction layer of intelligence” is central, as it represents the most valuable position in business history. Specialized vertical AI solutions like Harvey (legal) and Cursor (coding) demonstrate this trend, benefiting from highly documented and verifiable domains. However, scaling this to 1.5 billion knowledge workers will be challenging.
NVIDIA: The Central Bank of AI
Jensen Huang and NVIDIA are positioned as the “central bank of AI.” Their strategy of being vertically integrated yet horizontally open allows them to support a wide ecosystem of accelerators and customers. NVIDIA’s data centers are the most financeable, leveraging RVGs (Residual Value Guarantees) and low-cost capital to democratize compute access and foster competition. Jensen’s foresight in locking up critical supply chain capacity (fab, DRAM, NAND, lasers, capacitors) years ago gives NVIDIA an unparalleled advantage. Open source, far from being a threat, is seen as beneficial for NVIDIA, driving increased token consumption and thus demand for compute. The complexity and capital intensity of semiconductor hardware make it extremely difficult for competitors, emphasizing the importance of working within NVIDIA’s ecosystem rather than directly challenging it.