⚡️ The 2026 Software Factory Infra Stack — Zach Lloyd, Warp.dev
Zach Lloyd, founder and CEO of Warp, explains how software factories turn tasks into pull requests, measure cost, throughput, and quality through centralized agent traces and scorers, and improve themselves through benchmarks and self-improvement agents. He reports cutting Warp's cost per PR from about $80 to $15–20 while raising throughput, and argues that the factory should be defined, observed, and tuned as code.
Software factories as a concrete engineering system
Zach Lloyd frames a software factory as more than a loose collection of agents and automations. In his view it is an agentic system in the cloud where the roles of various agents are defined across building software for a team or product, with a deliberate amount of autonomy. The key requirement is that the system be measurable and closed-loop, so a team can tune, measure, and benchmark it over time and gain confidence that its move to agentic development is scalable and efficient.
The common misconception is that buying a tool produces a money machine. Lloyd argues that most companies are stringing together automations and calling the result a factory. His counter is to make the factory a specific artifact: defined in code, with its workflows, skills, and MCPs all explicit, runnable at a given point in time, and measurable. At Warp, each product surface gets one factory, operating on a set of repositories with defined skills, MCPs, and a workflow that mirrors how the team already changes its codebase, such as triage, spec, and implementation. The workflow is described in a YAML file that lists the repos, MCPs, and integrations in use.
Two personas and the work item metaphor
The factory serves two groups. Product engineers ship features by putting work into the factory through tools they already use: Slack, Linear, Jira, GitHub, an API, or MCP. Each work item enters a predefined, code-defined workflow. Lloyd’s point is that every task, whether it starts as a Slack message or an automated alert about a server error, flows through one system. That single point of entry is what makes the rest of the benefits possible. The second persona is the platform or DevX team, which uses the same data to tune how the whole organization gets the most out of coding agents.
Centralized traces as the basis for a closed loop
Once work flows through one system, the factory captures a full trace of every agent conversation and artifact, including computer-use screenshots and the orchestration agent’s steps. Lloyd emphasizes that companies today mostly have engineers using Claude Code, Codex, or similar tools locally, with no central record. Centralization turns those scattered sessions into data about team output and quality.
That data supports several kinds of measurement. Traditional DORA-style metrics show how long shipping takes. Automation metrics show how many times a human had to intervene per merged PR. Throughput and cost can be sliced by person, by automation, or by SDLC stage. Cost per PR is treated as central: the goal is to tune models and skills so that cost per PR falls over time.
Quality is measured through what Warp calls scorers, which are online evals. Some are general, such as whether agents are too verbose, too inefficient, or producing poor code. Others are specific to a customer’s failure modes, such as changes that break the build. Scorers are defined in code and can use an LLM as a judge, with a chosen judge model, a judging prompt, and a sampling rate to control cost. A concrete example is a scorer that flags redundant tests, a common failure where agents write tests they don’t need, leaving bloat that wastes tokens.
Agents improving the factory itself
Lloyd’s most interesting claim is that the factory can improve itself. Self-improvement agents look across scores for failures and suggest changes. In the demo, they found 54 failure cases in which the triage step would have done better by spawning a sub-agent for its investigation. Because the factory is defined as code, these agents can open a pull request against the factory definition. Lloyd says the next step is replaying past runs against a proposed change to confirm it would have improved them, which Warp does not yet do.
Benchmarks built from real work
Benchmarking means taking agent runs from your own system, curating them into a suite (for example, front-end tasks), and replaying those tasks across a matrix of configurations. Lloyd is clear that the variable is not only the model but also the harness: Warp’s own harness, Claude Code, Codex, or Pi can all be compared. The results resemble published SWE-bench or Terminal Bench numbers, but they reflect a company’s actual work. The output is a cost-versus-correctness trade-off, and the goal is the upper-left corner: low cost and high correctness. In one example, one model scored about 95% correct at around $6 average cost, while a smaller model scored 93% at about $2. Lloyd argues that such data can feed a model or harness routing strategy tailored to a team’s own codebase. He frames this as moving from guessing to what he calls factory engineering.
A layered architecture
Lloyd presents the factory as a stack, with the declarative definition at the foundation. Defining the factory as code gives it version history, branches for testing, revert, and the ability for coding agents to propose improvements, which he sees as the basis of a self-improving setup. He compares this to Terraform.
Above that sits a store for everything agents produce. Records need to connect across tools, so that a conversation starting in Slack, continuing into Linear, and ending in a GitHub code review can be traced back to the merged code and, if a server crashes, diagnosed afterward. Access control matters at scale: marketing, sales, and engineering may need different MCP access.
The compute layer should be pluggable. Warp uses Namespace by default, but the design should accommodate Daytona, Modal, E2B, or others. Linux sandboxes suit most work, but testing iOS apps requires a Mac VM that runs the simulator. Inference is also open: companies can bring their own provider, often one with existing commitments to Anthropic, OpenAI, or Google, or buy through Warp. Any ACP-compatible harness can be plugged in, including Pi.
The improvement layer sits on top of these, and the orchestrator moves work through stages, calls the right models, loads skills, and fires automations. Access points come last. Warp brings the factory into existing tools, with about 60% of its work starting in Slack, while some developers submit specs from their terminal. The system is built to be API- and MCP-driven, so it works as headless infrastructure with many entry points.
Concrete results and the economics of subscriptions
At Warp, the factory initially cost about $80 per PR, largely because Opus was used where a cheaper model would do, the triage agent ran on a high-powered model, and too many sub-agents were spawned. After internal tuning with these built-in features, cost fell to roughly $15 to $20 per PR. Over the last month, PR throughput rose about 40%, and the revert rate did not increase. Lloyd cites this as evidence for the approach he calls observe, measure, tune.
Lloyd also discusses subscription economics. Small teams often rotate across several $200 coding-agent subscriptions, which can be five to fifty times cheaper than paying per API token. He acknowledges this is not scalable and likely violates terms of service, and that it loses many of the benefits of a centralized system. He explains that Warp deliberately avoids competing as a pure coding agent, since subsidized subscriptions make that a losing fight, and instead positions itself as an optimization and productivity platform.
Who needs this, and where it fits
Lloyd sees the factory as overkill for a few people, who are better off just using the subscriptions and moving fast. The approach becomes relevant at roughly 5 to 10 engineers, and the value grows with scale. For fast-growing companies of 10 to 100 engineers using a common stack such as Slack, Linear, and Jira, adoption can be largely self-serve. Very large companies with bespoke infrastructure, such as Uber or Ramp, are poor fits, because they build their own systems and need substantial implementation work.
Product engineers and factory engineers
Lloyd’s upcoming AIE CODE talk focuses on the emerging split between product engineering and factory engineering. The open questions are what an engineer’s role looks like when the job is building the system that builds software rather than building the software directly, what tools and skills that requires, and how engineers develop them.