Your AI Agents Shouldn't Care Where They Run - Nikhil Harithas, Factory
Factory's Nikhil Harithas explains how Droid, their coding agent, was rebuilt to run on any compute inside a customer's own cloud. The talk covers the Open Compute abstraction, the compute gateway and outbound-only relay design, bring-your-own-key model calls, snapshots and forking for agent swarms, and the operational hard parts of managing compute for long-running agents in regulated enterprises.
Agents need their own identity and their own compute
Agents that run on a developer’s borrowed identity were a temporary shortcut. Once an agent is delegated work, such as a Jira ticket assigned to Droid that comes back as a pull request, two questions keep coming up in enterprise settings: who is the agent acting on behalf of, and where does it run. Factory’s view is that agents are slightly non-deterministic applications, so the answers should come from the same identity, networking, and infrastructure patterns used for decades of software engineering, repositioned for agents.
Why the defaults broke for regulated customers
The motivating case is Blackstone, a large financial institution with heavy regulation, strict security constraints, and firm opinions about deploying non-deterministic software. Running agents on laptops was out, and so were Factory’s default hosting and inference. The customer wanted to bring its own compute, its own inference, its own identity, and single-tenant deployment everywhere. Factory had earlier offered a bring-your-own-compute product, but installing an agent onto a machine was not enough, because the customer still had to manage the compute lifecycle, which turns out to be a hard problem.
Open Compute: abstracting the compute lifecycle
The answer is Open Compute, a project Factory plans to open source, built to abstract the compute lifecycle for any agent rather than only for Droid. The core unit of work is a computer, not a session. Tying the abstraction to a Factory session would make it Factory-specific and harder for a customer to trust. Customers run it on Kubernetes, EC2, Google Cloud, or whatever they already operate, and the compute layer is treated as its own plane that is spun out from the control plane.
A driver wraps a specific kind of compute. One driver might target EC2, another Kubernetes pods, and others could wrap sandbox providers such as Daytona or Modal. Open Compute talks to the driver to create, pause, resume, or destroy machines, and each machine is built from templates that define exactly what gets installed and which container or snapshot baseline it starts from.
Control plane, compute gateway, and relay
The request path has several layers. A client, such as the CLI, the web product, or a third-party integration, asks the backend to start a session. The backend calls a compute gateway, which sends the request to the Open Compute controller that already holds an open connection into the customer’s VPC. The controller uses the appropriate driver to spin up a machine, such as a micro VM, an EKS pod, or an EC2 instance.
The design is deliberately egress-oriented. Customers are nervous about ingress but much less nervous about egress, so once the environment is set up, Factory never makes an inbound connection. The agent daemon on the new machine establishes an outbound WebSocket to the relay in the control plane and keeps it open. Messages from the client then pass through the relay down to the sandbox, and the agent and client carry on a two-way exchange over that channel.
Bring your own key and keeping everything in the VPC
LLM calls can also stay within the customer’s boundary. For mature engineering organizations that already have an agreement with a model lab, the model traffic can go through their own key, for example via a Bedrock setup. The result is that much of what an on-prem deployment would offer is available without porting the whole control layer into the customer’s environment. The customer keeps observability and controllability over the non-deterministic part of the agent loop, and the blast radius of anything deemed too risky for an agent stays inside their VPC.
This matters beyond any one customer. Owning the stack, including evals, models, compute, and the product’s hosting, is likely to become a priority for large enterprises, and the goal is to make that practical.
Snapshots, forking, and the hard parts of managing compute
Snapshotting and forking are central patterns. As agents delegate subtasks to other agents, sessions fan out into fractal trees, so the ability to save a snapshot, fork it, and hand a subtask to another agent becomes essential. Snapshots can be stored in S3 or a customer’s database of choice and hydrated with secrets from their own secret store, so the customer decides how and where compute is built.
Managing arbitrary compute is messy. A request to start a sandbox may be acknowledged by the cloud provider without the sandbox ever materializing, or a request may drop mid-session without an acknowledgment. The system has to track what was started, route requests from each user to the right sandbox, and retry safely so the client and the sandbox agree on where they are in the conversation. Limits also matter: micro VMs in AWS carry an eight-hour session cap, while Factory’s missions can run multi-agent orchestrations for several days. That forces a way to snapshot a sandbox, rehydrate a new one, and swap it in place as if nothing happened. During private preview, distributing the deployment through versioned Terraform packages was another challenge.
One practical outcome is that the spec itself becomes the test. Droid can be asked to build a new driver that satisfies the spec, and if it does, the driver is effectively ready, which makes it straightforward for others to contribute drivers for new compute types.
The underlying point is that agents should not need to know where they run. Making compute a portable, customer-owned layer is how Factory delivers that flexibility across customer environments, with the open source release expected in a few weeks.