Building Data Centers for GPU Clouds

Craig Tavares, COO of Buzz HPC, provides an in-depth look at the complexities of building and scaling GPU cloud infrastructure for AI. He covers the critical role of renewable energy and strategic location, the evolution of data center design to handle extreme power densities, the importance of a strong partnership with NVIDIA, and the rise of sovereign mandates shaping the future of AI cloud services.

October 3, 2025 5 min read

Building Data Centers for GPU Clouds — source: https://www.youtube.com/watch?v=fj0mMeZA07w

Craig Tavares, with a deep background in building telecom infrastructure, data centers, and clouds, shares his perspective on the evolution and future of high-performance computing (HPC) infrastructure tailored for AI workloads. The modern AI landscape has fundamentally changed the design philosophy and geographic strategy for data centers.

The Foundation: Power, Land, and Location

The traditional model of building data centers was to place them in key markets close to large population centers—the so-called “NFL cities” in the US. However, the immense and continuous power appetite of AI model training has shifted this paradigm. The primary building blocks are now power and land. The strategy has evolved to build data centers where there is an abundance of cheap, renewable energy.

Key locations for Buzz HPC and its parent company, Hive Digital, include:

Quebec, Canada: Leverages massive hydroelectric infrastructure originally built for industrialization. This provides a sustainable and cost-effective power source.
Sweden: Another region with abundant green energy, primarily hydroelectric and geothermal.
Paraguay: A recent acquisition provides access to land connected to one of the largest hydroelectric dams in the Western Hemisphere.

This focus on renewable energy is not just about cost but also about fulfilling green mandates, as seen with battery manufacturers who also flock to regions like Quebec to power their energy-intensive processes sustainably.

The Evolution of Data Center Design and Cooling

The rise of powerful GPUs has created an exponential increase in rack density, forcing a complete rethink of data center design and construction.

Power Density Shift: Racks have rapidly evolved from a sub-10 kilowatt (kW) standard to 40 kW with the NVIDIA H-series GPUs, then to 80 kW, and now planning for 135 kW with the upcoming GB200s. Jensen Huang is already preempting the need for 400 kW racks.
From Air to Liquid Cooling: At these densities, air cooling is no longer viable. The thermal transfer efficiency of liquid is necessary to dissipate heat from the chips effectively. This requires massive plumbing infrastructure to move liquid at sufficient volume and speed.
Modular and Future-Proof Design: The pace of change is so rapid that infrastructure cannot be rigid. The philosophy is to “not screw anything down to the ground,” keeping designs modular to accommodate future hardware iterations.
Efficiency and Innovation: Data center efficiency is measured by Power Utilization Efficiency (PUE), with a goal of being as close to 1.0 as possible. Techniques to improve PUE include:
- Free Air Cooling: Using cool ambient air from climates like Northern Canada and Sweden for most of the year.
- Heat Reuse: Capturing waste heat and providing it to neighboring businesses in a “district energy system.”
- Deep Water Cooling: Using cold water from large bodies of water (like Lake Ontario for downtown Toronto) to run cooling loops.

The Software, Orchestration, and Hardware Stack

Building a reliable and performant AI cloud requires a tightly integrated stack, from the physical hardware to the software orchestration layer.

Partnership with NVIDIA

Buzz HPC has standardized on NVIDIA, viewing them as a crucial partner, not just a supplier. NVIDIA provides significant support to its cloud partners, including:

Reference Architectures: These create a consistent, reliable, and high-performance experience for customers. Standardization is not about rigidity but about establishing a proven recipe that can be updated with new innovations.
Hardware and Software Roadmaps: Early previews into future hardware (like denser GPUs) and software stack advancements allow partners to plan their data center designs and platform offerings accordingly.

The Orchestration Layer

Simply providing bare metal access to developers leads to significant waste, as users treat servers like “pets instead of cattle,” hoarding expensive GPU resources. A sophisticated software and orchestration layer is essential for efficiency.

Maximizing Utilization: As a service provider, the goal is to schedule workloads efficiently to maximize cluster usage and monetization. This involves offering different consumption models like Kubernetes or Slurm.
Customer Enablement: The objective is to abstract away the hardware complexity. By providing tooling and recommendations, customers can optimize their workloads, run on cheaper GPUs when appropriate, and extend the life and value of their projects. This is where partners like Rafé provide critical orchestration and automation capabilities.
Data and Storage: High-throughput storage is critical to prevent bottlenecks. Using solutions like Vast Data ensures that the network storage can keep up with the demands of the GPU clusters.

Sovereign AI and the New Hybrid Cloud

A major driving force in the global buildout of AI infrastructure is the rise of sovereign mandates. Nations recognize AI as a strategic imperative and are investing heavily to ensure they have domestic control over the entire stack.

What is Sovereign AI? It encompasses domestic control over the digital infrastructure, its operations, and most importantly, the data itself.
Government Strategy: Countries like Canada are appointing AI ministers and creating grant programs to fuel innovation, grow the domestic talent pool, prevent “brain drain,” and ensure government itself can be a massive, secure consumer of AI.

This trend is also giving rise to a more practical and necessary hybrid cloud model for AI. Unlike the traditional hybrid cloud, which was often more theory than practice, the AI-driven model is a necessity.

“You may stay in cloud or you may… create hybrid environments where, yeah, now I do edge into my own on-prem data center… And then I use cloud for bursting and I use cloud for different projects.”

Companies are finding that the unit economics work better when they own a baseline of hardware for high-utilization training while using specialized clouds like Buzz for bursting, inference, or experimenting with new projects. However, this is not without challenges, as data movement and egress fees can be significant hurdles if not architected correctly.