If Your AI Scores 100%, Your Eval Is Wrong - Ramakrishnan Lokanathan, eslams.com
Ramakrishnan Lokanathan argues that AI evals are broken because they are overcomplicated, and lays out a practical framework for building them for high-stakes applications: define the most ambitious version of success, test every step of a workflow, combine automated checks with human review and adversarial tests, guard against regressions, and turn real user failures into tests. He also introduces eSlams, an open-source package where models compete in games like chess, Go, and poker as a window into how they reason.
What an eval really is
Evals are broken largely because we overcomplicate them. At first principles, an eval is just an evaluation, and we already understand the idea from human exams like the SAT, the GMAT, and the bar exam. For AI systems, benchmarks such as SWE-bench and Humanity’s Last Exam play the same role. Every exam, human or machine, comes down to tasks, standards, and evidence used to judge whether something performs as expected.
A key distinction is that an exam answers a bounded question. The SAT measures language and math and is only one signal of college readiness. In India, admissions can hinge on a single exam, while the US treats the SAT as one signal among many. Model benchmarks deserve the same treatment. No single eval measures an agent’s complete abilities, and over-indexing on one, whether a humanities exam or SWE-bench, gives a misleading picture.
The real purpose of evals is to check whether performance matches the promise made to the user. Every product or agent makes an implicit promise, and evals exist to confirm that promise is being kept in production.
Why evals matter now
If model capability keeps improving, the need for heavy eval scaffolding will shrink over a decade. In the meantime, production cannot wait. Without the right eval structure, organizations fail to deliver what they promise. The MIT GenAI Divide report, which found that 95% of organizations were getting no returns from their generative AI efforts, is the cautionary example.
Five principles for high-stakes evals
Define the most ambitious success
Before designing any test, ask what the agent would do in the best possible case, imagining something close to AGI. Start from the job itself: what must happen, what must never happen, and when should the system stop. The guiding line is “aim at the job you need, not the score you can get.” A perfect score is a sign the eval is wrong. Top models currently sit around 85 to 86 on most evals, and nobody reaches 100, so an eval should stress the system rather than flatter it.
Test the whole workflow, not just the final answer
A simple image-editing example makes the point. The instruction was to change only the background, but the output changed the background to cream and also added two handles. Checking only the start and end states would miss that failure. In a workflow with a hundred steps, each step needs evaluation of what goes in, what comes out, and what outcome is expected. Tests should match the specific job of each step. A tool-calling step should be measured on tool-call accuracy, not on a generic process score.
One score is not the system
Early on, models were measured for coding or writing ability. Today’s agentic systems do many things at once, so the evaluation has to be holistic. Three layers matter:
- Automated checks wired into CI, so evals run every time a change goes through production deployment.
- Continuous human review. Agents can speed up the review process, but they should not be the sole reviewer of the evals.
- Adversarial tests that push the workflow into challenging situations to find where it breaks.
Board games have long been a strong measure of planning and long-chain reasoning. AlphaGo was a major milestone, yet general-purpose LLMs are rarely tested on Go or chess. eSlams is an open-source Python package on PyPI that lets models play games against each other. Chess and Go test planning under invisible state, poker tests decision-making under hidden information, cooperative games test shared goals, and arcade games add further variety. The package supports about 50 games and 90 providers.
The result of a match matters less than how each model plays it. A model that changes strategy, avoids blunders, and accounts for earlier moves is showing reasoning that a static benchmark score cannot reveal. One practical use is to have an old agent and an upgraded agent play each other before deployment. In one tic-tac-toe match between two frontier models, the game ended in a tie, which again shows that the way a game is played carries more information than the final outcome. A planned large-scale competition of AI models in February is meant to extend this idea.
Keep testing what used to work
Regression testing is essential because a much better model can lose a capability the old one handled well. Google Assistant’s move to Gemini is the example: reminders that worked reliably stopped working, and the new system responded that it was a language model that could not set reminders. Keep regression cases and past failures in the suite permanently.
The suite also has to keep moving forward. When models reach around 85 to 90 percent on an eval, it is time to make the eval harder and push the frontier. If nearly everyone scores in the 99th percentile on the SAT, the test itself has to change.
Turn real failures into tests
Users are the most important evaluators of an agent. Observability tools, thumbs-up and thumbs-down signals, and user reports surface failures in production. Each one should be treated like a classic QA bug: reproduce it, determine whether the failure came from a tool call, a reasoning step, or something else, identify which part of the eval suite should have caught it, and add a test to the pipeline. Pairing this feedback loop with consistent human review catches the last 5 percent of edge cases that a model or an existing framework would otherwise miss.
Putting it together
The core idea is to test the job, not just the answer. Define the most ambitious version of success, test every step of the workflow against what that step is supposed to do, and gather evidence from many paradigms, including coding, writing, games, and real-time retrieval such as search over live indexes, rather than trusting a single score. Keep regression tests alive, raise the difficulty as models improve, and let real-world failures become the newest tests, because reality is the most reliable teacher.