Why AI Compute Comes in Rectangles — Erik Bernhardsson, Modal
Erik Bernhardsson, CEO of Modal, traces his path from building Spotify's recommendation system to founding a serverless GPU cloud, and explains how Modal aggregates demand, secures GPU capacity, weighs renting against owning, and sees agent sandboxes becoming a major product line.
Modal’s Business and Origins
Modal builds AI infrastructure: a compute platform and storage platform with an SDK that lets teams like Decagon, Suno, Runway, and Cognition ship AI applications without managing capacity. Most of the workload is inference, with sandboxes, fine-tuning, training, and batch jobs alongside.
Erik Bernhardsson’s path ran through seven years at Spotify, where he built a music recommendation system from near scratch and shipped features like related artists and Discover Weekly, mostly through matrix factorization and other compute-heavy work. The frustration with infrastructure during that period became the seed for Modal. He later spent five years as CTO at Better.com, growing the tech team from roughly two to three hundred people, before leaving around 2020 or 2021 to take a loose sabbatical during the pandemic.
Modal’s lineage runs through Luigi, an open source workflow orchestration framework Erik released at Spotify. Around 2021 he returned to workflow orchestration and realized the harder problem was not orchestrating compute but executing it. That shift produced Modal, which started as a general-purpose way to run code in the cloud.
Finding Product-Market Fit
Stable Diffusion, released around the same time as ChatGPT, was the first real breakout use case. Modal’s developer experience turned out to suit production inference because autoscaling is closely related to the problem of quickly scaling containers up and down. Revenue went from about $100 a month in late 2022 to roughly $30,000 a month by January 2023, driven heavily by a single customer, which at one point accounted for 80 to 90 percent of revenue. Diversification followed, with Suno becoming a major customer in late 2023.
Erik credits much of the early growth to founder-led marketing: prolific posting, blogging, and building in public. The company self-funded for a long time, which kept it frugal, and raised a seed round in early 2022.
Rebuilding for a New Workload
Erik’s central thesis is that each technology era hits the limits of existing tools. Kubernetes and Docker were built for steady, low-utilization backend workloads where CPUs are cheap. GenAI inverts those assumptions: GPUs are expensive, models are large and must be loaded over the network into GPU memory, and demand is bursty. Modal therefore built its own file system, container runtime, autoscaler, scheduler, and snapshotting for CPU and GPU, rather than wrapping older tools. An early and consequential decision was to treat capacity as one global pool across regions instead of separate regional clusters.
GPU Supply, Renting vs. Owning, and Price Risk
Modal rented almost all of its capacity, initially spot and on-demand from hyperscalers, later one-year reservations, and Oracle for certain GPUs. That model broke down as demand grew and the market shifted toward three-year contracts requiring meaningful upfront payment. Modal now works with nearly 25 clouds, runs a dedicated procurement team, and has a roughly ten-person engineering group managing bare-metal capacity around the world, dealing with network issues, bad drives, and dropped GPUs.
On ownership, Erik is pragmatic rather than dogmatic. Renting offers capital efficiency, while owning brings operational and financial complexity: vendors, power, cabling, equity dilution, or debt that requires locked-in long-term revenue. He is skeptical that software companies should own hard assets, but says everything is on the table, including joint ventures or more unusual financing.
He sees current scarcity as a temporary state in which hardware prices rise, though hardware prices historically trend down over the long run. Scarcity helps Modal’s pricing power, and its reservations make it effectively long GPU prices. The bigger risk he worries about is a price collapse that would hurt buyers locked into expensive multi-year contracts. He compares the scarcity rent to economic rents captured mainly by Nvidia and, to a lesser extent, anyone owning capacity. He is cautious about buying at the top and points to the Hunt brothers’ silver corner as a reminder that cornering a market rarely ends well.
Modal’s supplier evaluation involves a spec sheet covering IOPS, CPU performance, and network bandwidth, acceptance testing on single nodes, and an internal reliability score that weighs responsiveness and turnaround on support issues. Smaller, less-known clouds often offer the best prices, because Anthropic and OpenAI prefer large, proven blocks for training runs.
Alternative Accelerators and Neoclouds
Erik has predicted the year of non-Nvidia accelerators for years and has been wrong each time, but he thinks 2027 may finally be it. AMD is improving on certain benchmarks, TPUs work well internally at Google, and Trainium’s software layer is the open question. Software is easier to fix than hardware, so better compilers and runtimes could unlock demand. He sees deeper bottlenecks at TSMC and ASML, and believes monopolies rarely last, so diversity would be healthy. He also finds Cerebras and Groq interesting, particularly disaggregated inference that pairs Nvidia for prefill with Cerebras for decode, since compute-bound and memory-bound phases can be mixed.
On neoclouds, Erik explains how they scale so fast: they sign multi-year contracts with a large buyer, use those contracts to raise debt or equity, then purchase hardware and deliver it. GPUs act as collateral, and this pre-selling lets them grow faster than a usage-based company like Modal can. Modal’s PLG motion makes that financing path harder, though he expects it to converge eventually.
On GPUs as an asset class, he notes startups such as Silicon Data and Compute Exchange are building price indexes and derivatives. He acknowledges compute is less homogeneous than commodities, but compares the situation to oil, where commoditized tradable instruments did emerge. He expects more hedging and insurance as AI infrastructure grows, but is unsure GPUs will become as liquid as Henry Hub natural gas or Brent crude.
Functions, Sandboxes, and Product Strategy
Modal has two core products. Functions, launched in 2021, are serverless functions that are easy to deploy and scale, and they now make up about two-thirds of revenue, mostly for inference. Erik stresses that Modal is not meant to be a neocloud selling rectangular three-year GPU blocks; its advantage is aggregating thousands of customers into smooth, bursty-friendly demand.
Sandboxes launched in 2023 after the team noticed that LLMs were generating code that needed a safe place to run. Early traction was small, and they nearly shut the product down. Then LLMs became strong at generating code, and vibe-coding companies, background agents, and reinforcement learning labs drove rapid growth. Sandboxes now account for roughly a third of revenue, and Erik says revenue keeps climbing by about 30 percent each month. Modal added full VMs alongside its gVisor-based sandboxes, because some workloads need full Linux compatibility.
Erik describes a future where sandboxes shift from long-lived dev boxes in the cloud to ephemeral environments: one per command, snapshotted between runs, with pause, resume, clone, and fork. He imagines long-running agents behaving like durable execution workflows that spin up sandboxes in around 100 milliseconds, run a few commands, and shut down. The median sandbox today lasts about eight seconds. He also imagines parallelizing a CI test suite across a thousand sandboxes, which current overhead makes impractical until startup and teardown friction falls.
He is also bullish on training for smaller and midsize workloads, such as speech models, computational biology, and recommendation systems, which are underserved because setting up training clusters is hard. On batch, he cites a customer transcribing millions of audio files, and expects low-latency voice workloads to grow.
On fine-tuning, Erik has noticed the value proposition shifting. Open-source models are now strong enough to swap in for frontier models at much lower cost, so fine-tuning is more optional, though Modal still sees real demand for SFT and RL.
Security, Agents, and Hugging Face
Modal’s sandbox security came into focus after a Hugging Face incident in which agents exploited a vulnerability in an app deployed on Modal to gain broader internet access. Erik stresses that the flaw was in the customer’s app, not the platform, drawing an analogy to AWS customers whose apps get hacked. He sees the broader lesson as making security more restrictive by default, comparing it to S3’s later tightening against accidental public buckets. Modal has also begun using agents internally to probe its own sandbox escape paths, and he believes agents will help defenders at least as much as attackers. He is less worried about cybersecurity incumbents than people might expect, arguing that enterprise relationships, brand, and the value of having someone to blame keep them relevant, while humans with AI tools become more capable.
Developer Experience and Agent Optimization
Erik argues that most things that make developer experience good also make agent experience good: clear docs, descriptive errors, and fast feedback loops. Agents have made difficult configuration tools like YAML-heavy systems less painful, but infrastructure is trending toward programmatic SDKs. A surprising problem is that agents trained on older Modal code keep producing deprecated syntax, so Modal has become more conservative about removing features and must update every public example. The company also runs internal benchmarks asking agents to deploy things like Stable Diffusion, checking whether Modal gets recommended, and it treats this as a form of SEO through content.
Forward-Deployed Engineers and Product
Modal has scaled its forward-deployed engineering team from roughly four to around 24 this year. Their work spans sales support, performance tuning, especially for LLM inference, cost optimization for customers over-provisioning CPU or memory, and migrations. Erik insists the team should not become a services business. Its most important function is surfacing product gaps that can later become self-serve features. He also says Modal has only recently started building a product management function, and has hired PMs sparingly because infrastructure product strategy is hard: it requires deciding the right foundational building blocks rather than shipping many similar-looking features.
Culture, Code Review, and Scale
Modal’s engineering culture leans toward individual contributors, and Erik says managers are no more prestigious than anyone else; everyone is a member of technical staff. Code review is optional at Modal. In his view, mandatory review often exists to dodge accountability for production changes, and AI-era teams should invest instead in CI, CD, tests, rate limits, decoupled systems, and resilience. Agents have also become good at writing tests, which he once thought missing. Modal expects roughly 2.5 to 3 times growth next year, potentially reaching about 700 people. Growth at that pace brings cultural drift, and he watches for speculative, fringe work as the team scales. Hiring is mostly systems engineers, plus people working on GPU performance and an emerging research team focused on post-training, distillation, and quantization.
Deployment Models and Capital Use
Erik remains a cloud maximalist and sees on-prem as a legacy model. Multi-tenancy lets Modal iterate quickly because it controls the full stack, while customer-deployed enterprise software constrains release management. He would consider bring-your-own-cloud for a very large customer at the right price, and sees BYOC as a possible way to work with neolabs that already hold large capacity reservations.
Capacity prepayments are the largest current use of cash, and Erik describes salaries and office space as a rounding error by comparison, given the operating leverage in the business. The three priorities he names are capacity and financial structure, the next product building blocks, and company culture as the team grows toward 250 people by year end.
Inference Providers, Model Builders, and Where Models Go
Erik argues there is a structural reason inference providers should be separate from model builders: demand for any single model is spiky, and a provider can reallocate GPUs across many models because aggregate demand is smooth. He is less convinced that LLMs are commodities, comparing frontier labs to artists who sustain brand and margin. He believes models will keep improving in verifiable domains such as code, math, and cybersecurity, and is more uncertain about areas with diffuse targets and longer feedback loops. He points to computational biology and materials science as promising fields where an LLM can propose candidates, simulations can score them, and an agent can close the loop.