Same GPU, 50% More Output: Why the Operator Matters — Oliver Kennon, SemiAnalysis
Oliver Kennon of SemiAnalysis argues that GPU hours are an input rather than the product, and that the operator determines how much useful intelligence those hours produce. The talk covers serving stacks, cluster security, ClusterMAX operator ratings, rental price versus production cost, relative cost-curve positioning, and what compute grades and futures can and cannot hedge.
Oliver Kennon, research analyst at SemiAnalysis, argues that buying GPUs gives you potential, but the operator decides how much useful work you actually get from them. He treats compute as a commodity market, where producers compete to deliver comparable output and the position on the cost curve determines who keeps margin when prices fall.
Compute as a commodity market
In a commodity market, producers lined up by what they spend to deliver the same usable work form a cost curve. If two producers earn the same price, the one spending less keeps more of the revenue. Kennon’s long-term North Star metric is price per successfully completed task at the required quality. Because that is hard to standardize, he uses price per token as a practical proxy for comparable workloads and quality requirements.
He separates cost from price. A GPU rental rate of dollars per hour is an input cost, not a measure of output. Hardware and electricity are the inputs, software and operations act as a multiplier on them, and the product is useful intelligence. The operator is the one who chooses and implements the software that serves the model and keeps requests flowing.
Same chip, roughly 50% more output
Kennon uses the InferenceX calculator on SemiAnalysis’s site to compare two serving stacks on B200 GPUs. Both serve the same model with about 8,000 input tokens and 1,000 output tokens, with a P50 target of 100 output tokens per second per user. In the P90 scenario, one stack produces about 200 output tokens per second and the other about 300. The difference comes from software, settings, and the number of concurrent requests, all operator choices. Assuming the hourly bill is identical and both stacks meet requirements, the better stack’s rental spend per acceptable token is proportionally lower.
What counts as useful intelligence
Useful intelligence must be good enough for the task, arrive quickly enough, be available when needed, and stay within agreed access boundaries. Kennon argues that security is an overlooked requirement that chip performance metrics do not capture. In SemiAnalysis’s published security work, they found monitoring access that exposed another customer’s logs and metrics. In that case the GPUs were very fast, and the service still failed badly.
Keeping a cluster secure, like delivering promised performance, is ongoing work, and the operator usually owns it. Deploying a faster serving update involves testing it on the customer’s workload, checking that outputs still meet requirements, load testing, keeping a rollback path, and then maintaining the system and responding to incidents. That work may sit with the provider, the customer’s team, or both, so the buyer needs to know who is responsible and whether they can actually do it.
Rating operators with ClusterMAX
SemiAnalysis’s ClusterMAX, now at version 3.0, assesses operating capability on managed GPU clusters. It covers a couple hundred clusters and is growing. The evaluation looks at provisioning reliability, monitoring, and security, drawing on hands-on testing, documentation, and customer feedback. The aim is to understand how a service behaves not only on a good day but also when a machine fails. Kennon stresses that ratings apply to managed clusters and do not carry over automatically to bare metal, where more operating work falls on the customer. Tiers are relative to peers. Pricing and availability matter as well, though they do not directly measure token output.
Rental price versus production cost
A GPU provider rents out hours, and an inference producer buys those hours and sells responses. The rent is revenue for the provider and an input cost for the producer. The producer’s full cost includes rent plus software, staff, and other services. If the producer owns its equipment instead, it spreads the investment over the equipment’s working life and adds running costs. What a producer charges and what it spends are separate numbers.
Better operations lower the cost of each acceptable token because the same paid hours yield more usable output. To measure this, count all spending over a period and divide by the output customers can actually use. Operations improvements cost money too: testing updates, deploying them safely, and keeping them running all take people and resources, and the extra output has to justify that spending. Kennon argues the advantage can persist if a team repeats a sound operating process as software and workloads change. A single good benchmark does not establish it. As he puts it, “You have to earn it every single day.”
Your cost position is relative
Kennon uses a hypothetical with made-up cost units, which he jokingly calls bars of gold. Your full cost per successful task falls from 10 to 8, while a competitor’s falls from 6 to 12. You improved, but you started as the cheaper producer and ended as the more expensive one. That gives the competitor room to lower its price, provided customers find the two services equally acceptable. Getting better does not by itself move you ahead.
He notes that scarcity hides these differences. When demand far exceeds supply, as he expects for the next two to three years, producers with very different costs can still find customers, and strong revenue makes cost improvements feel less urgent. He draws on historical perspective: every commodity has seen swings in supply and demand, and there is no obvious reason this cycle will be different. Businesses with leverage in particular need to build cost advantage while the sun is shining and capital is available, because that matters most for surviving a downturn.
Grading comparable compute
The same chip can produce different results depending on how it is operated, and two offers for B200 hours can include different services and leave different operating jobs with the customer. A grade describes a particular compute service and what its buyers can expect, so that buyers know which services are comparable. A grade does not make intelligence interchangeable across every task.
A rental price benchmark summarizes GPU hour prices within a defined market. It is not the producer’s production cost, and it does not measure the intelligence delivered. Its reference point depends on where prices come from: an advertised offer differs from a completed trade, and service scope, contract terms, sampled sellers, and the aggregation method all shape the number. Operating quality can influence prices even though the index does not measure it separately.
What compute futures can and cannot hedge
A futures contract sets a standardized term for future delivery or cash settlement, and cash settlement depends on a price reference. Buyers and sellers both face uncertainty about rental prices. Kennon’s argument is that rating the operator is an essential building block for a credible compute futures market. The rating helps define the grade, and the benchmark then has to track the rent that parties actually pay or earn.
He is careful about the limits. A rating does not by itself define a futures grade, establish a sound benchmark, or create enough trading liquidity. A rental price hedge can offset rental price changes, but it does not fix a failed deployment or guarantee that a customer pays. Charging more than the index does not automatically make a hedge bad. What matters is whether changes in the reference track changes in your rent, and unexpected breakdowns in that relationship leave real price risk.
Closing takeaway
The producer who can repeatedly meet requirements at the lowest full cost per successfully completed task is the one who can produce the intelligence you need at the lowest cost. Operating capability affects both what you spend and how much useful work you get from it, and your position depends on what competitors can deliver, so the operator matters across the whole cycle. In Kennon’s words, “A GPU falls to the level of the operator.” He closes by calling for a better-informed compute market and invites partnerships and hiring inquiries.