Dylan Patel on GPT-5’s Router Moment, GPUs vs TPUs, Monetization

The dynamics of the AI industry are shifting from a pure focus on model performance to the economics of compute. This transition is evident in recent model releases and the strategic maneuvers of major technology companies.

OpenAI’s Shift to Economic Efficiency

Recent OpenAI model updates, like GPT-5, represent a strategic pivot towards cost efficiency rather than a raw increase in model size or compute per query. The key innovation is a “router” that dynamically allocates compute resources based on the user’s query.

• Dynamic Compute Allocation: Simple queries (“Why is the sky blue?”) can be routed to a smaller, cheaper model, while complex, high-value queries are sent to the most powerful models. For power users, this can be a downgrade, as the average compute spent on their queries may decrease compared to previous models like o1 or o3 which would “think” for longer.
• Monetization Strategy: The router is a foundational piece for monetizing the vast base of free users. For low-value queries, OpenAI can minimize costs. For high-value, transactional queries (“Book me the best flight,” “Find me a DUI lawyer”), the system can dedicate immense compute resources to power an agentic workflow, taking a commission on the transaction. This model circumvents the awkwardness of placing traditional ads in an AI assistant.
• The New Benchmark: This move highlights that the new frontier for model competition is the trade-off between cost and performance, not just achieving the highest score on a benchmark.

The Value Capture Problem in AI

A fundamental challenge in the AI ecosystem is the “value capture problem.” While AI is already generating immense economic value, the model providers are capturing only a tiny fraction of it.

“I legitimately believe OpenAI is not even capturing 10% of the value they’ve created in the world already.”

This disparity is driven by the commoditization of inference, fueled by open-source models and efficient serving libraries. As the cost of intelligence decreases, it becomes harder for foundational model companies to maintain high margins on API calls alone. This reinforces the need for new business models, such as the agent-based transactional model.

NVIDIA’s Fortress: Why It’s Hard to Compete

NVIDIA’s dominance in the AI hardware market is not just about having the best chip; it’s a multi-layered moat that is incredibly difficult for competitors to overcome. Any new entrant must be significantly better, not just marginally so.

“You can’t just like do the same thing as Nvidia. You have to really leap forward in some other way. You have to be like 5x better.”

NVIDIA’s key advantages include:

• Supply Chain Dominance: Better negotiating power with TSMC, memory suppliers like SK Hynix, and every component manufacturer down to copper cables. This results in better costs and priority access to new technologies (e.g., next-gen HBM, advanced process nodes).
• Time-to-Market: They consistently bring new architectures and technologies to market faster than competitors.
• Ecosystem Lock-in: The entire AI research and development world operates on a hardware-software co-design loop optimized for NVIDIA GPUs. New model architectures are developed based on what runs efficiently on NVIDIA’s latest hardware. A competitor optimizing for a specific model type risks the world moving on to a new architecture by the time their chip is released.
• Software: A massive, mature software ecosystem (CUDA, Triton, etc.) that is the industry standard.

Even with a 5x hardware advantage on paper, a competitor would see that lead eroded by NVIDIA’s supply chain efficiencies, faster roadmap, and potential margin compression, making the final real-world advantage much smaller and harder to justify for customers.

The Competitive Landscape

1. Custom Silicon: The Biggest Threat The most significant challenge to NVIDIA comes from hyperscalers developing their own custom silicon.

• Key Players: Google (TPU), Amazon (Trainium), and Meta are investing heavily and increasing their internal orders.
• The Hyperscaler Advantage: They have a captive, internal customer, which means they can win simply by compressing margins and tailoring designs to their specific, high-volume workloads (e.g., Meta’s recommendation systems). They don’t need to compete on the open market.
• A Strategic Opportunity for Google: Google’s TPUs are highly competitive. It is argued that Google should consider selling them externally, which could unlock enormous value, potentially rivaling the market cap of their core business. However, this would require a significant cultural and organizational shift.

2. Silicon Startups: A Catch-22 AI accelerator startups face an uphill battle. They must make a disruptive technological leap, but this is fraught with risk.

• They often over-index on the dominant model architecture of the time (e.g., large, dense Transformers).
• By the time their chip is ready, the research community has often moved on to new techniques (e.g., smaller, more numerous matrix multiplies) that are better suited to NVIDIA’s more general-purpose architecture. They are trapped trying to predict the future while NVIDIA sets the pace.

Infrastructure, Not Capital, is the New Bottleneck

In the US, the primary constraint on AI expansion is no longer capital but physical infrastructure, particularly power and data centers.

• Power Scarcity: The main challenge is not the total amount of power generated but getting it to the right place. Grid interconnections, substations, and transmission lines have long lead times.
• Data Center Deficit: There is a severe shortage of data-center-ready sites with adequate power. This has led to companies spending huge sums to acquire crypto mining operations simply for their power contracts and physical locations.
• The Cost Equation: The vast majority (around 80%) of a data center’s cost is the capital for the GPUs and networking. Power and cooling are a relatively small fraction. This means it’s economically rational to spend exorbitantly on solutions that bring a cluster online even a few months faster, as the value of the compute time far outweighs the premium paid for rapid deployment.

In contrast, China has ample power infrastructure but is constrained by access to cutting-edge, cost-effective chips due to export controls.

Advice for Tech Giants

• Jensen Huang (NVIDIA): Use NVIDIA’s massive cash reserves to invest directly in the data center ecosystem. Accelerate the build-out of power and facilities to create more demand for your own chips.
• Sundar Pichai (Google): Become more aggressive. Sell TPUs on the open market, open-source the XLA software stack, and accelerate data center build-outs to reclaim the compute leadership position.
• Mark Zuckerberg (Meta): Continue the aggressive push on infrastructure and models, but also focus on shipping more standalone AI products outside of the core Meta ecosystem to compete directly with players like OpenAI.
• Satya Nadella (Microsoft): Address the product execution gap. Despite having premier assets—GitHub, a partnership with OpenAI, and the world’s best enterprise sales force—products like GitHub Copilot are being outpaced by smaller startups, and Microsoft Copilot is underperforming.
• Tim Cook (Apple): Recognize that AI is the next fundamental computing interface, replacing touch. The company must make a massive, multi-billion-dollar investment in AI infrastructure or risk being disintermediated as user experience shifts from the device to the AI agent.

• Tags:
• Feature
• Geopolitics
• Data Centers
• AI Economics
• Nvidia
• AI Hardware
• Custom Silicon
• Intel