Which GPU Clouds Are Actually Good? | ClusterMAX 3.0
A discussion of SemiAnalysis's ClusterMAX 3.0 ranking of managed GPU cloud providers, covering the evaluation criteria, the security problems found across neoclouds, the scarcity of compute as inference providers turn profitable, the strategic position of frontier labs that now resell compute, Nvidia's acquisitions of Poolside and Hugging Face, the separation between buyer-focused rankings and investment research, and the spread of heterogeneous silicon from Huawei and others.
ClusterMAX 3.0 and the State of Managed GPU Clusters
ClusterMAX started as a way to answer a question SemiAnalysis kept receiving after ChatGPT’s launch: founders who had just raised a seed round and needed compute wanted a quick, grounded read on which providers to trust. Rather than answering each request individually, the team published a tier list with a podium, with deeper consulting and due diligence available on top. Version 3.0 tracks 323 total providers, up from roughly 111 emerging neoclouds in the first release, and the evaluation has moved from hand-run benchmark scripts to an automated repository of industry-standard and custom benchmarks, reliability tests, and burn-in suites that coding agents can launch and debug largely unattended.
What a Neocloud Is, and What Is Being Tested
A neocloud is anyone renting out access to compute, but the term distinguishes AI-focused operators from hyperscalers and legacy clouds. The evaluation covers performance, security, reliability, ease of use, pricing, and availability across training, inference, and increasingly RL workloads. The criteria are deliberately scoped to the managed layer: managed Slurm, managed Kubernetes, and the provider’s willingness to hot-swap failed nodes. Bare metal and data center construction are explicitly outside the ranking, which explains why a company like Crusoe can be an excellent data center builder and still be downgraded for its managed cluster offering.
Security Is Still the Weakest Link
Security was described as the most alarming finding. The first generation exposed cross-tenant access, with testers able to see other customers’ jobs, data, and storage, because network security controls were not properly implemented. Later generations showed similar issues, though some providers improved after being called out. In one case, testers could see the workloads of national intelligence and military-style agencies running on a single neocloud. The checks themselves are deliberately simple: is the software current, are drivers up to date, is the cluster configured correctly. Running two- or three-year-old software with documented CVEs requires no frontier model to exploit. To address this, the team released a free CLI tool that checks software versions and points to upgrade paths. InfiniBand key configuration, such as M_Keys and P_Keys, is another area where the team finds deployments that are far from sound.
The Bar Keeps Rising, and Hardware Choices Signal Seriousness
Each generation has tightened the standard, so most movement has been downward. Google is the notable exception, rising from bronze to silver to gold. The bar for this round was the latest hardware, specifically whether a provider is deploying GB300 NVL72. Customers with $100 million to spend overwhelmingly want that system, but it requires a new network, ARM CPUs, Blackwell GPUs, 800 Gb networking, and mandatory direct liquid cooling, so nearly everything about the physical facility, OS deployment, and driver management changes. The ranking dynamic that emerged is practical: providers that deploy at scale and fix problems in an hour rather than two weeks climb, while those whose clusters break, whose support lags, or who report reliability problems slide. Reliability is consistently the criterion that top customers care about most.
A Market Where Compute Is Scarce and Labs Are Competing with Profitable Inference Providers
Availability has become severely constrained. Inference providers such as Baseten, Fireworks, Together, and Modal are running at gross margins around 60 percent on open models, which means anyone who can secure a GPU can immediately earn money from it. A neolab now competes against businesses that are profitable, whereas previously it mostly competed against loss-making labs. The minimum practical allocation has dropped: labs once rented clusters of around 8,000 GPUs from a single provider, and now purchases of 1,000 GPUs are common. Even four nodes can support a profitable inference business, though orchestrating across many clouds remains hard.
The largest buyers are OpenAI and Anthropic. Anthropic’s compute spans AWS Rainier, Google TPUs, CoreWeave, and more than ten clouds, along with its own leased and built data centers. Google’s position has weakened in a specific sense: it still holds more total compute than OpenAI, but much of that is sold to others, and by year-end both OpenAI and Anthropic are expected to have more compute dedicated to R&D than DeepMind. Google’s cost advantage from building its own TPUs remains real.
Labs Becoming Neoclouds and the Logic of Optionality
Labs that build data centers are increasingly renting out surplus capacity opportunistically, and they keep the option to pull it back for research or flagship training runs. The SpaceX and Google arrangement reportedly includes mutual 90-day cancellation rights, a structure that resembles a standoff over whose research needs will win. Meta, Mistral, and Poolside are following similar paths. Selling compute to others may reduce a lab’s margin, but the speakers argued that it funds research and lets the lab reclaim capacity when it matters most. On the Google question, the discussion acknowledged that the strategy of funding researchers who leave to found neolabs that then spend on GCP looks clever for GCP’s revenue but poor for Google as a whole. Switching clouds has also become easier, because AI can help migrate workloads and a GPU is largely fungible across clouds.
Poolside, Hugging Face, and Nvidia’s Defensive Acquisitions
Poolside’s team, which built much of its own infrastructure after the neoclouds disappointed it, was acquired by Nvidia in a deal reported at $12 billion, though the true split between licensing, employees, and investment is disputed. The argument is that the value Nvidia paid for is talent with real experience training models, and that proven experience running training clusters is rare. Hugging Face fits the same pattern. Nvidia’s free cash flow of roughly $200 billion a year makes these purchases affordable, and the deals keep influential companies from falling into rival hands. The discussion also highlighted that Poolside’s infrastructure work, including health checks and high reliability for long training runs, is exactly what a frontier training cluster requires. Few providers have demonstrated they can run a 100,000-GPU cluster; by the speakers’ count, perhaps five companies.
Addressing Criticism and the Separation of Ranking From Investment
The most common objection is that the rankings reflect financial interest. The team states plainly that it takes no payment for rankings, which would be illegal if undisclosed, and that its value depends on telling the truth. Dylan disclosed personal investments in Crusoe and Fluid Stack and noted that neither company ranks in the top five. He also noted that many large infrastructure buyers, including AWS and some large Gulf customers, are paying clients without affecting the rankings. The team also distinguishes the rankings from institutional stock research. A provider can rank poorly on managed clusters while owning a gigawatt-scale data center that is a strong investment, and the two analyses intentionally diverge. One example was Nebius, which the institutional side favored over CoreWeave because its shorter contracts capture more upside as GPU prices rise, even though CoreWeave is still the better cloud. The rankings are meant for buyers, not for choosing equities.
The team is extending the scope beyond managed clusters. Upcoming work includes Endpoint X for serverless inference providers, evaluated on cost, reliability, security, and monitoring rather than raw throughput, and Inference X for chip and system comparisons. Because inference performance on a benchmark can differ sharply from stability and reliability in production, the team argues that a complete picture requires all of these dimensions.
Silicon Diversity, RL Workloads, and the China Question
The next year will bring more heterogeneous silicon. Nvidia’s Vera Rubin generation may include LPUs from Groq, and AMD, Cerebras, Trainium, and various startups such as Positron, Etched, and MatX are all pushing into the market. The boundary between training and inference is blurring. RL now involves large volumes of forward passes, and on that basis mixed silicon is attractive for cost and throughput. But numerical correctness becomes a challenge when chips differ even across generations, such as B200 and B300, because memory sizes and batch behaviour diverge. The team expects homogeneous clusters for pre-training at very large scale, while heterogeneous hardware gains ground in RL. Distributing RL updates across geographically separated sites is possible in principle but faces difficult synchronization problems.
On China, Huawei’s Ascend line is improving quickly, with the 940, 950, and 960 released in rapid succession. Huawei is the only Chinese manufacturer producing over a million chips a year, though its chips still trail Nvidia’s. Other domestic companies, including Cambricon, Iluvatar, and Enflame, are producing in the hundreds of thousands. The team is pursuing access for testing, specification review, and reverse engineering. As open-source models improve at writing kernels, porting becomes easier, and CUDA’s lock-in weakens. That pressure, the speakers argued, is why Nvidia now pushes for annual chip and system releases.