Inside the $41B AI Cloud Challenging Big Tech | CoreWeave SVP
Corey Sanders, SVP of Product at CoreWeave, explains why the unique, high-stakes demands of AI workloads are driving a shift away from general-purpose clouds toward specialized "Neo Clouds." He details the specific hardware and software innovations in storage, cooling, and networking that allow CoreWeave to maximize GPU utilization and deliver superior performance, arguing that this focused approach creates a durable competitive advantage.
Corey Sanders, SVP of Product at CoreWeave and a former Azure executive, posits that the AI revolution has created a new tier of business-critical workloads that general-purpose clouds are ill-equipped to handle. This has paved the way for specialized “Neo Clouds” designed exclusively for AI.
The Case for Specialization
Corey argues that when a workload is both high-cost and critical to the business, customers will seek best-in-class solutions rather than settling for a “jack-of-all-trades” platform. He draws a parallel to the analytics wave that gave rise to companies like Snowflake and Databricks. The fundamental limitation of public clouds is their need for fungibility—their business model relies on serving a vast array of diverse workloads, which prevents them from making the deep optimizations AI requires.
“The lifeblood of a public cloud is fungibility… It’s very, very hard for these sorts of specializations to be built in.”
In contrast, a specialized cloud can make simplifying assumptions to build a superior stack. The core design principle is to maximize GPU utilization by ensuring the GPUs are constantly fed with data. As Corey puts it, the goal is “bringing as much possible data and throughput into the GPU as you can… because GPU is the most expensive asset.”
CoreWeave’s Technical Differentiators
Corey highlights several areas where this focused approach yields a significant performance advantage:
- Storage Architecture: CoreWeave developed a unique object storage and caching layer (Lada Cache) specifically for the read-heavy patterns of AI training. It is designed around multi-GPU deployments to optimize data delivery, a stark contrast to general-purpose caches that must balance read/write performance for applications like e-commerce.
- Liquid Cooling: The most powerful modern GPUs require liquid cooling. A specialized provider can design its data centers around this from the ground up, enabling large-scale deployments that are difficult for general-purpose clouds to retrofit into their fungible infrastructure.
- Networking Flexibility: For many AI workloads, particularly inference, the majority of the processing time is spent inside the GPU. This reduces the relative impact of network latency, creating opportunities for more flexible and resilient workload scheduling across geographically distributed data centers. This helps customers manage capacity bursts and improve availability without sweating the details.
- Observability and Tooling: CoreWeave provides deep observability into the hardware stack, which is integrated into tools like
sunk(Slurm on Kubernetes). This simplifies the management of complex training jobs and helps customers achieve higher “goodput.”
Customer Engagement and Product Philosophy
Corey emphasizes that CoreWeave’s product roadmap is driven by deep, hands-on engagement with customers. New features often originate from solving a specific, high-stakes problem for a single customer before being productized for a wider audience. This close partnership, combined with a technically superior product, fosters significant customer loyalty.
He also shares his personal product development philosophy, which involves actively building projects to understand user pain points firsthand. When seeking customer feedback, he advocates for showing a concrete prototype or idea rather than asking open-ended questions. This approach, he argues, yields far more actionable insights.
Ultimately, Corey believes the advantage of specialized clouds is durable. While the performance bar will constantly rise, the focus on a specific workload allows for continuous innovation that general-purpose clouds, bound by the need for fungibility, cannot match. As he concludes, “The level of quality, performance, and capability and experience that we deliver today will not win workloads in 2 years… we better be moving the bar up.”