No, That's Not a Software Factory — Ryan Cooke, WorkOS
Ryan Cooke of WorkOS argues that software factories should be judged by customer outcomes rather than PR counts or AI-generated code volume. He describes how WorkOS embedded its product engineering process into its factory through TARS, Horizon, a PM agent that drafts Hilltop docs, and an internal MCP gateway connected to Snowflake, along with bug triage, owned sandboxes, a memory layer and outcome-focused metrics.
Ryan Cooke, an engineer at WorkOS, starts from the now-standard software factory pattern: a sandbox, an AI coding agent such as Claude Code or open code, a prompt, a generated PR, and a merge. He credits Ramp’s Inspect, blogged several months earlier, with sparking the industry’s interest in building software this way. WorkOS built a version of this too, but found that its results were hard to tell apart from engineers running Claude Code on their own laptops. For Cooke, the lesson is that counting PRs or lines of AI-generated code is the wrong yardstick. Those output metrics can hide whether the work is driving anything, since a rising PR count does not show that the team is shipping more to customers.
Measuring outcomes instead of output
Cooke argues that a software factory needs dedicated engineering time, and that the organization has to justify that investment. The question to ask is whether the factory is actually accelerating feature delivery. His vision is that the factory gives each engineer a small team of their own, which only works if the team builds and ships more, and builds complex features faster.
From code generation to product engineering
The first WorkOS sandbox was built on Cloudflare with an open code model router, and it was not enough. Cooke says it “wasn’t driving more outcomes” than laptop-based agents. So the team decided to embed its engineering process in the factory itself, so the factory would handle product work as well as code. The system is split into two parts.
The first is TARS, the agent users interact with. It lives in Slack, Linear and GitHub, and subscribes to webhooks so it can track project progress as well as generate code. The second is Horizon, an infrastructure orchestration layer that sits in front of an internal MCP gateway. Cooke says that, by feeding webhooks from source control and project tracking into the system, the factory moves from doing code work to doing product engineering work.
A concrete example is dependency-linked tickets in Linear. When a ticket completes, TARS receives the webhook and picks up the next blocked ticket in the cycle. The team also asks TARS to re-evaluate a project between steps and flag missing tickets, because “as you do work, you’re learning about where the gaps in your plan” are. In this way the agent keeps the plan current.
Hilltop docs and the PM agent
WorkOS has no product managers on teams. Engineers own product functions, and their main ritual is the Hilltop document, a PRD covering the project’s purpose, customer signals, competitive analysis, early design screens and milestones. Because this process was already codified across the company, it could be handed to an agent. A dedicated PM agent drafts the Hilltop doc from a brief, adds context, reads human reviews, and then breaks approved work into tickets. Once a Hilltop is approved, the work starts automatically with no human shepherding each step. Humans can still comment or refine at every stage, and Cooke frames the agent’s main value as solving the cold-start problem.
In the demo, Cooke kicks off a project for a new Vaults API from Slack with a few sentences of description. TARS creates the Linear project, the Notion doc, the decision log and the open questions. Cooke then steps in as lead product engineer to add specifics, and hands the project back for execution. He admits the agent often overestimates scope and has to be cut back, but argues that editing a seeded document is far simpler for an engineer than creating every primitive from scratch. As a result, many teams now write more briefs and Hilltops, because the blank page is no longer the obstacle.
Agent choice and the MCP gateway
The factory does not require one coding agent. Some engineers point Devin at the same tickets and documentation, and others use local Claude Code with Opus, pulling context through MCP. Security reviewers also join project channels to weigh in on implications.
The internal MCP gateway, which WorkOS calls its context engine, connects all internal systems and also supplies system prompts that explain how to navigate each tool and when to use it. Snowflake, the main data lake, is a key example. Semantic tables describing product usage and customer conversations are exposed to agents through tool descriptions, so an agent knows what each table contains and how to query it. Cooke says the gateway turned out to be far more useful than expected. Internal teams now query it directly from Slack for data and customer analysis, and WorkOS is building further tools on top of it. His advice to anyone starting a factory is to invest early in an internal MCP gateway that connects tools, describes them to agents, and documents how the organization structures its information.
Triage, own sandboxes and memory
Bug reports arriving in Slack are picked up by TARS through webhooks, which triages them, drafts a first pass at a fix and opens a PR. TARS also helps triage customer support requests from shared Slack channels by reading the code to locate where a customer is likely hitting a problem.
WorkOS is moving away from sandboxes-as-a-service toward owning its sandbox infrastructure, mainly for deep control over session data and the ability to move workloads across its infrastructure. The next piece is a company-wide memory layer, an evergreen context covering what each person works on, their team and products, and the organization’s semantics. The goal is to use this both inside the factory and in other AI tools.
Success metrics and self-improvement
Cooke says the team watches defect rates and time to recovery, since it is concerned about introducing instability. It also tracks adoption qualitatively, looking at whether engineers choose to move from local harnesses into cloud sandboxes and whether they rely on TARS. Because WorkOS owns the infrastructure, it can inspect sessions and use them to improve the factory. The team looks for gaps where an agent made a mistake and a new skill is needed, for skills that have become obsolete as the codebase changes, and for places where engineers could improve. Cooke notes that the AI landscape moves so fast that this continuous verification matters.
He closes by repeating that sandboxes alone are not the point, and that the real aim is to encode engineering practices and processes into automation. He also invites the audience to talk about authorization, a problem WorkOS has not solved yet for its own factory.