Please Stop Telling Your AI to “Search Harder”
A discussion of how agent-driven traffic overwhelms APIs, gateways, and cloud infrastructure built for predictable load. It compares microservices with agent workloads, revisits SaaS multi-tenancy, and examines Agent Substrate's actor and worker model for suspend-and-resume agent sessions, with a critical view of whether most users will ever need this layer directly.
Agents Are Not Predictable Humans: Rethinking Infrastructure Under Agentic Load
The basic premise is that a single prompt can now fan out into dozens of searches, API calls, and compute-heavy requests, and most of the infrastructure between users and services was never designed for that pattern. Agents are now a large share of website visitors, and a human asking an assistant a simple question can put a load on systems that would previously have required a crowd.
Why Agent Traffic Breaks Existing Assumptions
Traditional machine users were predictable. Month-end financial reporting jobs or Black Friday rushes could be planned for because they repeated the same routine on a schedule. Agents are different: they improvise their route on the fly, issue variant requests, and do it faster than any human. The speaker’s point is that the unpredictability is not just larger in volume, it is a different kind of unpredictability, and the only real throttle on agents is the human decision to deploy them.
This affects far more than databases. Free public APIs, third-party search services, and even computer-use agents that drive a browser or desktop directly all absorb the spikes. In financial services, portfolio optimization and capital-market scenario testing are inherently compute-intensive, and an agent running what-if assumptions (oil up, oil down, property market crash or boom) can fire off far more of them than a human advisor ever would.
API Gateways and the Collapse of Request-Based Pricing
API gateway contracts were typically sized by forecasting annual call volume from past patterns. That model no longer holds. Usage can balloon without any real growth in customers, simply because agents scouring the web hit endpoints as one step among thousands in a larger workflow. The speaker has long been skeptical of per-request pricing in middleware, and argues this shift makes that skepticism harder to dismiss: a vendor cannot predict its customers’ usage when those customers cannot predict their own.
Microservices Versus Agent Workloads
Kubernetes replaced monolithic deployments by managing scheduling and scaling for workloads, and it served microservices well. Microservices, however, were typically many instances of the same service. Agent sessions are each distinct: each carries its own context, connectors, authentication, and project state. At the scale of a platform like a vibe-coding tool, with millions of users each spawning their own coding agents, the number of concurrent, unique, long-lived workloads outstrips what Kubernetes’ API-server-centric model was built to configure and manage. Kubernetes remains the substrate, but an additional management plane geared toward identical-yet-individualized agent workloads is needed.
Multi-Tenancy, Context, and the Cost of Warm State
The conversation draws a useful parallel to SaaS tenancy. Single tenancy, with fully isolated stacks per customer, is simple but expensive, since an occasional user keeps a whole instance running. Shared multi-tenancy reduces cost but requires carrying identity and authorization through every request. Agents add a further wrinkle: context grows large over time. Someone returning to a project after a month wants it loaded instantly.
The answer borrows from older performance techniques. Financial analysis software historically pre-cached data in memory, and Gmail’s login flow famously fetched a user’s inbox server-side while the user was still typing a password. Humans are predictable enough, and slow enough at typing, that the system has time to prepare. Agent platforms can apply the same idea by anticipating when a session will be needed.
Agent Substrate: Actors, Workers, and Suspend-and-Resume
Agent Substrate is described as an agent workload orchestration layer that runs on top of Kubernetes. Its core model maps a large set of actors, meaning agent applications that sit idle most of the time, onto a smaller pool of ready workers. Actors are not tied to a particular runtime; they are assigned to workers in real time. This allows heavy multiplexing of agents onto shared compute, and it supports multiple sandbox technologies, including microVMs and gVisor, with consistent lifecycle operations across them.
The promised capabilities are sub-second suspend and resume, persistent working memory across hibernation cycles, and fast reactivation onto any available worker. Applied to a vibe-coding product, a user returning to a project after a month could have their code pulled and the environment restored almost immediately.
The speaker is cautious about maturity. The project appears to be in beta or alpha, and early adopters have reported a lack of upgrade guidance between releases. Questions remain about how much it actually solves for fast memory loading and avoiding cold starts, and the speaker’s current understanding is that its focus is on making agent sessions resumable without cold starts rather than purely improving compute utilization.
Who Actually Needs This Infrastructure?
The most important question raised is whether most agents need cloud-scale orchestration at all. Developers in the coding-agent bubble tend to assume every agent needs scalability, triggers, and sharing. For most users, an agent is simply a prompt in a chat window, such as a parent using a phone assistant to plan an outing. Scale at the consumer end is high, but the number of companies that must operate large agent fleets is small. Vertical markets are naturally bounded: there is a finite number of registered financial advisors in the United States, while individual consumers number in the hundreds of millions.
The likely path is that the few providers operating at scale, along with the wrapper platforms built on top of them, absorb this complexity. Engineers and end users may use Substrate-like systems without realizing it, the same way many sophisticated technologies become invisible implementation details. Whether it proliferates beyond technical circles, and whether the complexity it adds is manageable, will determine if it becomes a lasting part of the stack.