Without Caching, Your AI Agent Costs 10x More - George Cameron, Artificial Analysis
George Cameron of Artificial Analysis explains what drives the cost of AI agents in 2026, covering input tokens, cache hit rate, reasoning effort and the number of steps per task. He walks through a worked example where a task costs about $21 with caching and over $200 without it, and offers a framework for deciding when a frontier model is worth its price and when a cheaper model delivers the same result.
Benchmarking the full AI stack
Artificial Analysis is an independent benchmarking company that covers the whole AI stack: agents (including coding and other agent benchmarks), language models, image, video, speech and music models, and the hardware and inference layers. The central argument is that these layers are interrelated. Choosing a model is also an inference decision: how you access it, what you pay, what speed you get, and what cache hit rate you can achieve. That is why the company’s results are often presented as two-by-two tradeoff charts rather than single leaderboard bars, since production rollouts depend on tradeoffs.
The intelligence landscape
The Intelligence Index synthesizes ten evaluations spanning knowledge work, scientific reasoning, coding and terminal use, hallucination, and other factors developers weigh when choosing a model. It is designed as a decision tool for the applications being built now, not as a measure of progress toward AGI. Anthropic’s Opus 5.5 currently leads it, but with more than 500 models tracked, the key takeaway is that the most intelligent model is not always the right one. Individual evaluations matter too. The AA-Analyst finance and accounting benchmark shows that Gemini models are strong at navigating complex spreadsheets, and Anthropic models are similarly strong there, a nuance the generalist index does not fully capture.
Progress is accelerating and release cadence keeps rising, which makes keeping up exhausting but worthwhile for anyone who wants to stay on top of what AI can do. Open-weights models trail the leading proprietary models by roughly 3 to 9 months, so they remain a real option, and their cost profile is often favorable.
Why cost has become the binding constraint
Frontier models now cost several dollars to complete a single agentic task, something that was not the case 12 months ago. At the other end of the spectrum, some models complete tasks for 5 cents or less, a difference of multiple orders of magnitude. The cheaper models are still very capable: Luna, at about 7 cents per task, is far better than GPT-4 was at launch and well beyond GPT-3.5 Turbo, which opened up the wave of language model adoption. Roughly two orders of magnitude separate the leading proprietary models from these much cheaper, still strong options.
Cost also moves over time. Once a given intelligence tier is reached, the cheapest way to access it falls by about 10x within a year. A score of 30 on the index first cost a bit over a dollar, fell to around 10 cents a year later, and now sits in the 1 to 10 cent range. Extrapolating, a spend of $1 million on a given capability could drop to around $100k and then to around $10k within two years. Decisions should be made for today’s prices while anticipating how quickly the curve is moving.
Breaking down the cost of an agent task
Reasoning models made cost analysis more complicated 12 months ago, and agents have made it far more so. Using the AA-Briefcase agentic workbench, where agents operate in environments of thousands of files on knowledge work such as reports, PowerPoints and marketing videos, Opus 5.5 averages $21 per task. Each individual model call costs about 13 cents, but the task takes 165 steps, so the number of steps is a new multiplier that agent builders have to account for. Within a single call, answer tokens and reasoning tokens are a relatively small share, and the larger drivers are input tokens and the proportion of them that hit the cache.
The number of turns per task is the most significant factor overall, while input token volume and cache hit percentage are the key levers on each call. The effect of caching is dramatic: if the cache hit rate is zero, which happens when prompt caching is not enabled or not working properly, the same task costs over $200 instead of $21. Cached input tokens are priced at roughly a tenth of the normal input rate. Artificial Analysis has begun publishing cache hit percentage benchmarks for this reason, since it matters for inference providers as much as for builders.
Reasoning effort and its compounding effects
Reasoning effort affects more than the reasoning tokens in a single call. It also changes the number of turns, and the reasoning content feeds into message histories, inflating input tokens across the whole agent trajectory. The same model can therefore vary enormously in cost: Opus 5.5 costs about $1 per task at low reasoning effort and $21 at max.
Comparing Opus 5.5 with GLM 5.3 Flash, an open-weights model, illustrates the broader point. Every one of the cost drivers contributes to the difference between models, and in long-horizon agent tasks, input token prices and cache hit rate are the dominant factors.
Making the tradeoff decision
Knowing that one model is cheaper than another is not enough. The real challenge is locating a given task on the Pareto frontier of intelligence versus cost per task. If a team can spend more, it should look toward the high-intelligence end of the frontier; if budgets are tight, it should look toward the cheaper end and ask which model offers the best intelligence for a given dollar.
A useful way to frame this is to sort tasks into two buckets. Unlimited intelligence tasks are those where more intelligence always yields a better result, such as analyzing the cost drivers of a business and deciding how to optimize them. For these, the question is simply what budget is acceptable for additional intelligence. Limited intelligence tasks are those where more intelligence does not improve the outcome. Asking how much was spent on subscriptions last month is a good example: Luna, connected to the right MCP servers, returns $80, and so does Astra, so paying for the more capable model buys nothing. For these tasks, the goal is to identify the ceiling of intelligence required and the cheapest way to reach it.
A practical workflow
The recommended approach is to understand the landscape and metrics, compare models against your priorities, shortlist candidates, and then test them in your own context and with your own agents. For small prototypes, a shortlist and quick testing is enough. For production rollouts, where cost magnitudes are significant, it is worth building a custom benchmark, narrowing to around five models, and optimizing for the specific decision. Artificial Analysis’s Optima product supports this by taking your context and generating a benchmark together with your own tradeoff curves between the intelligence your task needs and the cost of delivering it.