Why We Don’t Need More Data Centers - Dr. Jasper Zhang, Hyperbolic
Dr. Jasper Zhang argues that the relentless construction of new data centers is an inefficient, expensive, and unsustainable solution to the AI compute demand. He proposes a global GPU marketplace as a superior model, designed to aggregate fragmented, idle resources, drastically reduce costs through efficient allocation, and ultimately democratize access to AI infrastructure for developers and startups.
The explosive growth of AI is creating unprecedented demand for GPUs and data centers. Projections show a need for four times more data centers by 2030, with demand growing from 55 gigawatts to a projected 219 gigawatts. However, simply building more data centers is not a viable long-term solution. The current approach is plagued by fundamental inefficiencies that a smarter allocation of resources can solve.
The Flawed Data Center-Centric Approach
Continuing to focus solely on building new data centers presents several critical challenges:
- Prohibitive Cost and Time: A large-scale data center can cost over a billion dollars to build. Furthermore, the logistical hurdles are immense, with wait times as long as seven years just to connect a new facility to the electrical grid in key regions like Northern Virginia.
- Unsustainable Energy Consumption: Data centers and GPUs already consume 4% of the total electricity in the United States, contributing significantly to energy strain and CO2 emissions.
- Persistent Supply Deficit: Even if construction proceeds at the maximum possible rate, the US is projected to face a data center supply deficit of more than 15 gigawatts by 2030. This indicates that building alone cannot bridge the gap between supply and demand.
The Core Problem: Massive Underutilization
The more significant issue is not a lack of hardware, but its inefficient use. The AI compute space is highly fragmented, with over 100 GPU clouds, making it difficult for users to find and access available resources.
According to Deloitte, GPUs in enterprise settings sit idle 80% of the time.
This widespread underutilization means that a vast amount of computational power is wasted. The solution is not just to create more supply but to efficiently aggregate and distribute the existing, fragmented supply.
A Marketplace Model for GPU Allocation
A global GPU marketplace, acting as an aggregation and orchestration layer, can address these inefficiencies directly. By connecting idle GPUs from various providers—data centers, smaller clouds, and enterprises—to users in need, a marketplace creates a more liquid and efficient system.
Hyperbolic’s Implementation: HyperDOS
Hyperbolic, for example, uses a proprietary software called HyperDOS (Hyperbolic Distributed Operating System) to create this network.
- Functionality: HyperDOS is a Kubernetes-based agent that can be installed on any cluster. Within five minutes, it integrates the provider’s compute resources into the global Hyperbolic network.
- Architecture: The system operates on a “monarch-baron” model. A central “monarch” server receives user requests for GPUs and dispatches them to “barons” (compute providers), who then provision the machines and provide SSH access to the customer.
- Flexibility: This model allows users to rent compute in various ways, including spot instances, on-demand, long-term reservations, and managed model hosting.
The Benefits of Aggregation
This marketplace approach yields significant advantages founded on principles of queueing theory (specifically, the M/M/c model), which demonstrates that aggregating demand and supply drastically improves efficiency.
- Drastic Cost Reduction: By creating a unified distribution channel, a marketplace can reduce GPU costs by 50-75%. For example, an H100 GPU on the marketplace can be priced at $0.99/hour, compared to $2-3/hour on specialized clouds or as high as $11/hour on major cloud providers.
- Simplified Procurement: Founders and developers are freed from the frustrating process of vetting numerous suppliers through multiple sales calls. A unified platform with transparent pricing, ratings, and performance benchmarks streamlines the acquisition of compute.
- Enhanced Productivity and Flexibility: The true value extends beyond cost savings. A flexible marketplace model allows for a 6x increase in productivity for the same budget.
Use Case: Scaling Flexibly
Consider a startup that needs 1,000 GPUs.
- Traditional Model: The startup reserves 1,000 GPUs for a year. If they need 1,000 more for a one-month experiment, they are forced into another expensive, long-term contract. If they later only need 500 GPUs for inference, the other 500 sit idle, yet they continue to pay for them. The estimated cost for this inflexible scenario could be $43.8 million.
- Marketplace Model: The startup rents 1,000 GPUs. They can flexibly add another 1,000 for just one month. When their needs decrease to 500, they can release the idle 500 GPUs back to the marketplace, monetizing them and helping others access cheaper compute. The total cost in this scenario drops to $6.9 million—a 6x saving.
This empowers startups to train their own models rather than relying on closed, proprietary AI systems, thus democratizing AI development.
Conclusion: A Sustainable Future for AI
Focusing solely on building more data centers is an unsustainable path. The future of AI infrastructure lies in smarter allocation and the reuse of idle resources. A GPU marketplace provides a more efficient, cost-effective, and environmentally sustainable model that recycles compute power, reduces waste, and ultimately accelerates innovation by making powerful AI tools accessible to everyone.