Why AI Makes You 100x Faster but Not Your Company - Mitchell Troyanovsky, Basis
Mitchell Troyanovsky, co-founder of Basis, explains why individual engineers see huge AI productivity gains while most companies do not. He argues agents must be simple and reliable to spread through an organization, and that this requires opinionated agents governed by behavior specs. He also covers why outcome evals fail to generalize to production, and two diagnostic tools: forking and sub-agent simulations.
Why company productivity lags individual gains
Engineers and knowledge workers using tools like Claude Cowork, Codex, or Claude Code have seen dramatic personal productivity gains, yet the average enterprise has not seen a comparable jump. Mitchell Troyanovsky, co-founder of Basis, argues the reason is structural: agents have not yet been designed to fit how organizations actually scale work.
How organizations already handle non-determinism
People are non-deterministic, and organizations have long coped with that. The head of an accounting firm has no view of the day-to-day work of employees at the bottom, yet trusts the outputs because processes are put in place. If leadership believes a process will generalize to good outcomes, the non-deterministic people executing it will generally reach good outcomes. This holds even for small steps. Reconciling cash starts with a process for pulling bank data, such as using approved software, rather than letting each person choose their own extraction method.
Two gaps in coding-style agents
- Competency gap: Current agent products push complexity onto the user, who must manage skills and memory and decide when to invoke the agent. This requires a kind of theory of mind over the tool, which most knowledge workers do not have.
- Process gap: When each user customizes how the agent behaves, the way work gets done becomes an individual choice. That runs against how organizations scale, since process should come from senior leadership.
Solving these means agents must be simple and reliable. Simple means working like a co-worker: you state what you want, and everything you expect to work does. Reliable does not mean perfect. It means doing things the same way consistently, and improving over time the way a person does.
Being opinionated
Reliability requires opinions. Consider reading a PDF. The options are effectively unlimited: a VLM, open-source OCR, specialized OCR providers, or several VLMs checking each other’s work. An unconstrained instruction to extract a PDF may produce good results, but not reliable ones, because the approach is not fixed.
Opinions come from intuition and expertise. In tax work, having the agent read the primary source before answering makes it more accurate. Having an agent screenshot a PowerPoint before returning it reduces formatting errors. Opinions are not only about accuracy, though. They also trade off latency and cost. If formatting errors occur one time in a hundred and that is acceptable, the screenshot step is not worth its latency. The point is not which specific opinion you hold, but that you have one.
Why outcome evals don’t generalize
Traditional outcome-based evals have a right answer and a rubric. A set of a hundred or a thousand cases cannot represent the hundreds of thousands of situations an agent meets in production, and the number of decisions per trajectory grows as agents take on longer horizons. Most trajectories have no ground truth. Instead, teams can inspect trajectories to check whether an opinion is actually showing up in behavior. If it is, the behavior is more likely to generalize. This is not proof, but it is a meaningful signal.
Behavior specs
Take bank reconciliation. When the books do not tie, should the agent post an entry to force a match, or explain the difference? The right answer is debatable, but what matters most is consistency. A customer should not get one behavior one time and another behavior the next.
Behavior is governed by the combination of the system prompt, progressive disclosure, tool descriptions, and model training, not by a single paragraph. The context an agent sees while working functions as runtime training data, so every token shapes behavior, and the word ‘prompt’ understates this.
A behavior spec is written before the context is changed. It states the intended behavior, works like a product spec for the agent era, and captures the most important opinions without duplicating state across the system. It serves two purposes. It aligns the team, and it acts as a rubric: for any trajectory, including those with no ground truth, you can ask whether the agent encountered a situation where the behavior was relevant and, if so, whether it exhibited the behavior. This scales supervision far faster than writing more outcome evals. Outcome evals remain important, but they cannot be scaled nearly as fast.
Forking to diagnose bad decisions
Forking copies an agent at a point in its trajectory, similar to the fork icon under each message in Claude Code or the Codex web app. When a trajectory fails a behavior eval, the team can go to the moment just before the decision that violated the behavior, fork there, and ask the agent why it acted that way. Alternatively, they can update the context and check whether the behavior improves. Recreating databases and prompts perfectly in the cloud is hard, but forking turns a behavior failure into a concrete diagnosis that can feed harness improvements. An outcome eval offers no such path. When a five-hour run contains around a thousand inference calls, a failed outcome alone does not reveal why.
Simulations with sub-agents
Simulation is the speaker’s favorite tool. Ask a coding agent to create a sub-agent that pretends to be the production agent, instructing it not to cheat and giving it the same system prompt, skills, and rendered context. Then present it with a situation from an eval or from production and see whether it reproduces the observed behavior. If it does, tweak the context and check whether the simulated behavior changes.
Simulation is less faithful than forking, but it is much cheaper and faster, and it requires no infrastructure. The speaker used it for a year and a half before Basis built forking into its internal systems. Together, these tools address the central question of generalization. Outcome evals alone cannot show whether an agent generalizes across the distribution of trajectories, but supervising the process provides additional signal to answer that question.
Earning trust like a coworker
When reviewing a pull request, engineers weigh two things: the blast radius of the change and how competent the author is. An opinionated agent starts narrow, is easy to use, and proves reliable. As people see it work, they extend it more trust, just as they would with a new colleague.
Why not let agents find a better way?
Agents might discover better approaches on their own, but unconstrained optimization is not how organizations scale output or GDP. Nobody wants their agent to make a Move 37 on their tax return. They want work that is reliable, scalable, and trustworthy, and that their people know how to work with.
For agents to diffuse through a company, they need to be simple. For teams to trust them, they need to be reliable. Developing opinions about behavior is how you get there, and those opinions generate the supervisory signal needed to close the loop toward self-improving agents in production.