Don't Build Agents You Can't Answer For — Addy Osmani
Addy Osmani discusses the evolving role of software engineers in the age of AI agents, emphasizing a shift from code production to human judgment, accountability, and system ownership. He highlights new challenges like cognitive debt and the "orchestration tax," proposing that the modern engineer's value lies in "answerability," taste, and discerning which paths warrant human investment and responsibility.
Addy Osmani explores the transformative impact of AI agents on the role of software engineers, arguing for a fundamental shift from mere code production to human judgment, accountability, and system ownership. The core idea is maintaining the “human in the loop,” where the engineer of the future will primarily choose what is worth doing, owning the evidence, understanding, and the ultimate “verdict” for automated work.
The Shifting Landscape of Engineering
Osmani highlights that traditional craft boundaries are blurring, with roles “rebundling around the work itself.” The key question for engineers is no longer about their title but rather “what part of the system can you own?” While AI agents assist across all engineering modes (prototype, build, sweep, grow, maintain), the scarce skill becomes discerning which mode is needed, what quality bar applies, and who owns the result.
He discusses the evolution of AI in software development:
- Harness Engineering: The coding agent is the model wrapped in a “harness” (context, tools, file system, Git), transforming intelligence into delegable tasks.
- Loop Engineering: Systems continuously prompt, check, remember, and decide next steps, making agents feel like infrastructure.
- Software Factories: These combine the above, with agents handling the “inner loop” of execution, while humans retain critical production decisions. Human judgment, in this context, becomes the highest-leverage checkpoint.
Answerability and Review Bottlenecks
With AI-generated and assisted code becoming commonplace, “answerability” shifts from a philosophical concept to an engineering requirement. Osmani notes that cleaner code not only benefits human readability but also improves agent efficiency (fewer tokens, fewer revisits). A significant challenge arises because while generation becomes cheaper, review does not. Survey data suggests widespread skepticism of AI code (96% distrust), yet only about half of engineers consistently verify it before committing. This creates a dangerous “distrust without bandwidth,” where review and validation become organizational bottlenecks as AI adoption outpaces governance and policy development.
Critical questions emerge:
- Did a model touch this file?
- What constraints guided its work?
- What evidence was produced?
- What risk was accepted?
- Who owned the result?
The Value of Human Judgment, Alpha, and Taste
When agents can ship more code than humans can review, the scarce resource becomes “judgment backed by evidence.” Osmani introduces two terms: “alpha”—the gap between human capabilities and current model capabilities—and “decay”—the rate at which this gap closes. He argues that capabilities like speed, recall, and verification are subject to rapid decay as models improve. While “taste” (the ability to make high-quality qualitative judgments where objective metrics are absent) might decay more slowly, it is not an eternal moat. The best form of taste involves making better calls and providing clear examples for systems and teams to learn from. The strategy for engineers is not to cling to any single capability but to continuously “move our edges up a level.”
Therefore, the strategic question shifts from “What can the agent do?” (which has an ever-shrinking list) to “What can only a human be answerable for?” This involves decisions requiring ownership, context, risk acceptance, and responsibility after the work ships. An engineer is defined not by coding ability alone, but by the capacity to reason about systems, understand constraints, defend trade-offs, manage risk, and be the point person when systems break.
Risks to Avoid
Osmani identifies three key risks for engineers in this new paradigm:
- Cognitive Debt: The erosion of understanding and memory about problem-solving due to over-reliance on AI. This leads to “delegation debt,” where a team can ship a working system but lose the ability to explain it.
- Cognitive Surrender: Blindly accepting AI’s responses without independent judgment. Citing a Wharton study, Osmani warns of “borrowed confidence,” where individuals can feel more certain even when AI provides a wrong answer.
- Orchestration Tax: The false assumption that running more AI agents in parallel means more human cognitive bandwidth. Every additional loop creates more decisions to route, merge, and verify, demanding a deliberate design of human attention.
Accountability, Credibility, and High Agency
Accountability is presented not as a relic of pre-AI engineering but as the foundation for scaling complex systems. While agents can do more work faster, the scarce abilities become explaining intent, inspecting evidence, accepting risk, and improving systems when decisions are wrong. Osmani draws a distinction in “career math”: the “half-life of an edge” (capability) is short, tied to model releases, but the “half-life of a signature” (credibility, expertise) is much longer. Skills earn leverage, but accountability builds trust.
Agents can execute, route, and escalate, but they cannot inherit consequences. Humans are responsible for understanding policy, accepting risk, and owning the “blast radius” when things fail. “High agency” means actively taking ownership of outcomes, knowing when to delegate, inspect, stop, and put one’s name on the result. This involves “ownership with judgment attached,” moving up a ladder from merely flagging problems to demonstrating “discernment”—deciding which problems are even worth solving.
The Human-Agent Boundary and Future Outlook
The operational boundary is clarified: agents handle the inner execution loop (investigate, implement, test, report—this is capability), while humans manage the outer loop (deciding, verifying, approving, owning—this is agency). The agent provides evidence, but then “engineering begins” as humans decide its worth, verify its sufficiency, and approve/redirect what reaches production. The key operational rule is “Explain it or don’t ship it.” This doesn’t mean typing every line, but someone must understand the work well enough to defend it and be accountable for its risks.
Osmani concludes with optimism: automation historically has not reduced demand for software but moved the “floor” up, unlocking “latent demand” for previously unfeasible ideas. AI agents will continue this trend, shifting the bottleneck from “can we build this?” to “should this exist, and can we answer for it?” The new work for engineers will be in loop design, evidence design, and brownfield stewardship, demanding more taste, verification, ownership, and care, rather than less.