AWS, AI-Native Startups, and the Infrastructure Rebuild for Agents

Opening and Scale of AWS

Matt Garman opens by framing AWS’s business as still early, with revenue around $169-170 billion and growth of about 37%. He points to a large remaining base of on-prem workloads that have yet to migrate, combined with tailwinds from both AI and cloud migration, as reasons the business keeps compounding. Garman also recalls interning at AWS in 2005, when the project was still internal, and his assignment was to identify which customers would find it most interesting. The answer was startups.

Startups as the Lifeblood of AWS

Garman argues that startups have been central to AWS since launch, for three reasons. First, the value proposition is strong for early-stage teams. Second, AWS invests in helping them with architecture, scaling advice, and getting companies running. Third, startups are “the enterprises of tomorrow.” He estimates that roughly 30 to 40 percent of AWS revenue comes from companies that were once startups. He also stresses that startups are where AWS learns: they push the envelope on what they want from services, often ahead of banks, healthcare companies, and governments, which will want those capabilities years later.

What has changed is the starting scale. Where a startup once raised around $10 million and iterated slowly on an app idea, today a team can be valued at a billion dollars from day one with hundreds of millions in funding. Garman notes that ambition demands more capital, especially for training models. Some things have stayed the same: founders still care about architecture, security, performance, and identity and access management (IAM) setup well before they have many employees, which is why many prefer AWS over smaller neoclouds.

Rethinking Cloud for Agents

A recurring theme is that teams increasingly want a cloud that works well for agents as well as people. Garman says AWS was already well positioned because its APIs are well defined and traversable by agents, and it has doubled down on properties like fast provisioning (starting a database in about three seconds) and broad scale. He distinguishes between services built specifically for agents, such as AgentCore and Bedrock, and existing primitives like S3 that have been optimized to serve both humans and agents. He mentions AWS Context, in preview or beta, which builds a context layer so agents can find data across stores such as Aurora and S3.

A notable insight is about tail latency. Humans often tolerate the occasional slow S3 request at the P999 percentile, but agents, which chain many calls, get blocked by it. Garman credits this focus on performance as a reason agentic workflows run better on AWS than elsewhere.

On developer experience, Garman observes that most customers now tell their coding agent (Kiro, Claude, Codex) their credentials and let it deploy. Users who are not yet on any cloud may go through partners with simpler layers. AWS has also made account creation much easier: new users can sign up with a Gmail account, without a credit card, and get a fully functional account in under 30 seconds, with VPCs and IAM roles handled by default. Crucially, this is a real AWS account, so users can later add organizational structure without migrating.

Garman also describes the rethinking required for transient workloads. Many agents want to create a database, do some work, and discard it, but production Aurora is built for five nines of durability. AWS is working through how to offer quick, cheap, disposable resources without a non-durable option that causes problems if a database turns out to be long-lived. He also points to brand-new building blocks agents need: compute sandboxes, gateways, and agent-specific permissions. Agents need time-boxed, short-lived, fine-grained permissions rather than a copy of a human’s access. Garman notes that Firecracker, AWS’s microVM technology invented about a decade ago, is widely used by sandbox startups because it spins up rapidly, has a strong security boundary, and carries little virtualization overhead.

The GPU Allocation Problem

The conversation then shifts to GPU access, a problem every startup faces. Frontier labs consume available GPUs aggressively, yet AWS wants to keep the broader ecosystem healthy. Garman says the capex for compute is enormous, with about $220 billion planned for 2026, a larger annual expense than almost any company has previously carried. Constraints shift over time: power, data center construction, capital, memory, chips, and construction labor. He notes that the company is working on multi-year planning, thinking about 2026, 2027, and 2028 needs simultaneously.

AWS’s allocation strategy serves frontier labs such as Anthropic, OpenAI, and Meta, as well as large enterprises like Salesforce and JPMorgan that need fewer accelerators but want Trainium or NVIDIA hardware. Garman says AWS could sell every GPU to the big labs but chooses not to, reserving capacity for startups. He says AWS says yes in some form to about 60 percent of requests, sometimes later, in a different region, or in a different configuration. He also announced that AWS will buy 2 million NVIDIA GPUs over the next couple of years, while acknowledging that power and other components will still limit how much can actually be deployed.

Is the AI Capex a Bubble?

Garman addresses bubble concerns directly. AWS’s capex has accelerated meaningfully over the last three to four years, and the company was historically investing ahead of demand. A painful side effect is that elasticity in core compute and storage has largely disappeared as GPUs absorb capacity. He argues AWS is less exposed than neoclouds, where one or two customers can represent 30 to 60 percent of capacity. At AWS, the largest single customer is in single-digit percentages. Most usage is core compute, storage, and inference embedded in applications, which he believes will persist because customers report positive ROI.

He draws on the venture model: not every billion-dollar startup will succeed, and the investment pays off when a few winners emerge, as happened with the internet, where companies like Google and Amazon endured. Garman is careful to say that AWS plans its investment with discipline and is bullish on the long-term potential of the market.

He also describes how planning has changed. Fifteen years ago, AWS asked a power company for a few tens of megawatts. Now it brings its own power, paying for renewable and nuclear projects, combining behind-the-meter arrangements with grid-connected credits. He notes AWS has been one of the largest renewable power purchasers for a decade. Garman argues that customers cannot plan power or memory footprints years ahead, so this planning is a service AWS provides on their behalf.

On bottlenecks, Garman references the book The Goal: there is never one constraint, only the latest one. He says the monthly question has been whether there is enough power, and once power eases, memory, TSMC capacity, HBM, networking components, or connectors may become binding. Location matters too, since power in Indonesia does not help in Germany. AWS tracks tens of thousands of components, some managed by suppliers and others directly, and it began thinking through the supply chain four or five tiers deep roughly a decade ago. He recalls the Thailand floods that wiped out disk drive supply as an example of why such planning matters.

Data Center Myths and Public Perception

When asked whether the industry has done a poor job explaining why data centers benefit America and the world, Garman agrees and says AWS needs to be more vocal. He points to renewable energy commitments, water positivity, AWS data centers using very little water through mostly free-air cooling, and high-paying local jobs. He is self-critical about the industry, noting that a few bad actors who ignore regulations harm everyone by association. He cites a county where residents pay about $5,000 less in taxes because of AWS’s tax contributions, a fact the community often doesn’t know. His broader point is that much of the data center community is responsible, and the job is to highlight who is a good citizen.

Custom Silicon: Graviton and Trainium

Garman traces AWS’s custom chip history back to Nitro. Around 13 or 14 years ago, AWS noticed a significant virtualization tax, and customers wanted bare metal performance. The first step was offloading network virtualization to a card. Then AWS acquired an ARM-based offload-card company and asked whether that hardware could also handle storage virtualization. That led to a bigger card that took network virtualization off the host, giving customers bare-metal servers with no hypervisor overhead and stronger security isolation. AWS can truthfully say it has no access to customers’ VMs.

From there, the team turned the ARM cores on the offload card into a server, and Graviton was born. Garman explains that AWS saw the trajectory of ARM core performance and power efficiency and concluded the architecture would soon make sense for mainstream workloads, so it needed to drive the ecosystem forward. Graviton is now a runaway hit: AWS lands more Graviton servers each year than any other type, and it offers about 20 percent lower cost with about 20 percent better performance. Over 90 percent of the top 100 customers use Graviton in some form, and some have halved their server counts after migrating.

About five or six years ago, AWS saw AI compute rising and built its first Trainium chip. Trainium 3 is now in market, with Trainium 4 announced. Garman says capacity is sold out through roughly the end of next year, and AWS is trying to carve out capacity for startups. Most Bedrock traffic runs on Trainium, and AWS has deals with Anthropic and OpenAI to build on it, plus half a dozen to a dozen smaller startups. He admits AWS is “terrible at naming”: Inferentia was for inference and Trainium for training, but as models grew, inference moved onto large systems, and Trainium turned out to be arguably the best inference chip on the market in absolute performance and cost-performance. Its architecture differs from competitors’ designs and is much cheaper. Large customers are also interested in Trainium for training clusters, and Garman says the Trainium 4 architecture is seen as the future for those very large training clusters.

Where Enterprises Are Stuck on Agents

Most enterprises have built relatively simple agents that remain non-autonomous, with people in the loop. Garman says they still get significant value, but are now trying to make agents autonomous and safe. He identifies two obstacles.

The first is conceptual. Enterprises often replicate an existing workflow step by step and have a human check the output at the end. Garman argues the better approach is a green-field rethinking: an agent can parallelize work, try 50 approaches, and reach a result in a fundamentally different way. He says startups already think this way, and the lesson is to help enterprises, including conservative ones like insurance companies, apply it.

The second obstacle is trust. Enterprises need guardrails, correct permissions on data, and assurance that an agent won’t delete a production database. Garman calls this nervousness appropriate, and AWS is working with customers on architecture and new technologies to address it.

He identifies evals as a central gap. Enterprises need continuous testing loops, data labeled in a way that makes evals meaningful, and production measurement with backtesting to catch drift. He says few organizations are good at this yet, which is why forward-deployed engineering (FDE) teams have proliferated at AWS and its partners. AWS’s FDE philosophy is to teach customers to do the work themselves: over roughly 45 days, engineers build evals alongside the customer, teach labeling, and then leave, so customers are not dependent on an external workforce.

Open Weights, Bedrock, and Data Privacy

On whether enterprises should post-train open models on their own data instead of sending everything to a frontier model, Garman agrees with the premise that enterprise data is the most valuable asset. He explains that Bedrock was built with a guarantee that customer data never leaves the VPC, and model providers never see prompts. He acknowledges that AWS was criticized for being slow to the AI world, but says it deliberately built the foundations to protect customer data. He says Bedrock’s growth is explosive, with OpenAI and Anthropic workloads both growing there.

He also describes AWS’s expanding support for open weights models, noting that many customers believe their proprietary data could help distill a better, cheaper model. Most such work today happens in SageMaker, which hosts the inference too. He says SageMaker, historically a model-building platform, is effectively becoming the home for enterprises building custom models, and AWS wants to make evaluation and testing across open weights models easier.

Risk, Security, and Continuum

CEOs, Garman says, mostly want to know how they can trust agents to do what they intend. AWS is investing in permissions, sandboxing, guardrails, and deciding when a human should remain in the loop. He addresses the worry that powerful models could be used for attacks, calling it a real risk but also an opportunity. AWS recently launched Continuum, which uses powerful models to scan a customer’s environment for vulnerabilities, prioritizes them using context about permissions and compensating controls, and is very popular with customers. Garman argues that security eventually needs to operate at machine speed, not human speed.

Agents Inside AWS

AWS uses agents across the business. Continuum reflects its own security practices, and AI is heavily used in software development. AWS rolled out Amazon Quick to every employee. HR teams now build agents that compress weeks of team planning into a couple of hours, and finance teams build agents that pull tax rules from many sources to support compliance. Garman says line-of-business employees can now unblock themselves rather than waiting on developers.

The biggest gains are in software development and product development more broadly. AWS’s frontier teams have moved from traditional development to agent-first development, where agents write the code and humans manage a team of agents. Garman says the pace of new product rollout has accelerated markedly, and customers have an almost insatiable appetite for new capabilities.

Organizational Implications

Garman does not claim to have the answer on how organizations should change, but expects that for a long period, there will still be many people working alongside agents. He describes experiments with smaller pods. A product team that once needed ten people to own a capability may now need three or four, and those people can build so quickly that they move on to new problems. The challenge is operating and maintaining what was built while staying agile across an organization as large as AWS. Employees, he says, enjoy the change because they can build faster and do more.

Closing Reflections

The interview closes with Garman thanking the host and expressing appreciation for the startups that AWS treats as customers and as teachers. The overarching thread is that AI is reshaping cloud infrastructure, from the startups at the frontier and the GPU allocation decisions that serve them, to the custom silicon, the rethinking of primitives for agents, and the enterprise challenge of trusting autonomous systems.

• Tags:
• AI Agents
• AI Security
• Feature
• Enterprise AI
• AI Infrastructure
• Cloud Computing
• AWS
• Custom Silicon
• Graviton
• Firecracker
• GPU Allocation
• Trainium
• Bedrock
• Data Center Capex
• SageMaker