Why Humans Can't Keep Up With AI-Written Code - Tariq Shaukat, Sonar
Sonar CEO Tariq Shaukat argues that agents have made code abundant but confidence scarce, and draws on Toyota's quality-first factory principles to show how to build software factories enterprises can trust. He covers five hard problems that break agentic development and presents Sonar's code confidence formula: intent, guide rails, verification, and evidence.
From the factory floor to the software factory
Tariq Shaukat, CEO of Sonar, starts with the 1980s car industry, when well-engineered cars still did not fit together cleanly. Toyota and Lexus changed that with a simple demonstration: a ball bearing rolled across the seams of a car. The lesson went beyond one commercial. Factories began to prioritize quality over volume, fix the system rather than individual symptoms, design for assembly instead of hoping parts fit later, solve problems as they occur, and empower everyone on the line to stop it. Shaukat argues that modern software practice inherited these principles through Agile, Kanban, error budgets, and continuous integration.
Why agents break the factory
Agents have made code creation cheap and abundant. A recent study found lines of code up 2,450% and commits up 240%, yet releases rose only 30%. Teams have not yet learned to scale from producing code to shipping it. Sonar’s developer survey shows the appetite for change: 5% of developers say they run agentic software factories today, 34% expect to within a year, and 39% in financial services expect the same. The same capabilities that let agents build software also let them find and exploit vulnerabilities, which makes containment and governance urgent. Shaukat points to Satya Nadella’s writing on insider risk from models inside companies, and his call for deterministic system design, human controls, and reliable operating procedures.
Five hard problems
Human review does not scale
Most organizations answer governance questions with ‘we have humans in the loop,’ but that does not hold at agentic volume. Shaukat describes cognitive surrender: a recent study found people agree with AI when it is right 93% of the time, and agree with it four times out of five when it is wrong. Speeding up review does not fix this. The model of human review itself needs to change.
Agents struggle with large codebases
In a code translation study, agents achieved 91% accuracy on a 10,000-line codebase, but accuracy collapsed by 50,000 lines. Token consumption on the same task also varied by 30x across codebases, because agents spend much of their budget learning the structure before doing useful work. Shaukat compares this to starting a trip from White Plains versus Albuquerque: the starting point determines the cost of the journey.
Errors compound over long tasks
METR’s research measures agent performance by the length of human tasks they can complete. The latest Claude Mythos preview handles tasks that take humans 16 hours, but only at 50% reliability. Raising reliability to 80% flattens the curve sharply. The arithmetic is unforgiving: at 95% accuracy per step, a 20-step task succeeds only about 36% of the time.
LLM judges are unreliable
Using an LLM to evaluate another LLM is a natural fix, but the evidence is weak. Research from the PayPal AI research lab found that between 7% and 17% of verdicts changed across identical replications, depending on the model and situation. A Meta study found that simply challenging the judge with ‘are you sure?’ flipped its answer between 7% and 34% of the time. An evaluator that changes its mind on repetition cannot be the sole basis for trust.
Time to exploit is near zero
Attackers now move faster than most organizations can respond, so prevention has to be designed into the factory itself rather than bolted on afterward.
The code confidence formula
Sonar’s answer is a framework built on four components: intent, guide rails, verification, and evidence. Shaukat frames them as multiplicative. If any one is missing, confidence drops to zero, even if the other three are strong.
- Intent: If you cannot state clearly what you are building, the agent cannot build it.
- Guide rails: Shaukat deliberately distinguishes guide rails from guard rails. Guard rails stop a vehicle from leaving the road; guide rails keep it on the road through small, continuous nudges. Agents need the second kind of governance.
- Verification: It must be independent, thorough, and result in fixes rather than deferred issues. Shaukat notes that Sonar began as a verification company and argues for combining deterministic analysis with agentic methods, using CPUs where they suffice and GPUs where they are needed.
- Evidence: Outcomes must be backed by repeatable, auditable proof. Shaukat lists examples of the controls organizations must satisfy, including IT general controls, MISRA for automotive software, and other quality gates. In his words, evidence is what lets you sleep at night.
Results from customers
For long-running tasks on large, complex codebases, Sonar Vortex, which supplies intent and guide rails to agents inside the agentic loop, reduced token cost by 36% and issues in commits by 92%. The effect is the same as giving the agent a map and placing it near the answer.
Across more than 1,000 SonarQube customers, users were 81% more likely than non-users to report a very positive impact from AI coding on code security, 34% more likely to report a positive impact on time to market, and 76% more likely to report a positive impact on outage frequency.
Shaukat’s closing point is that code is abundant but confidence is scarce, and the factories of the future must be built to deliver confidence.