Practical tactics to build reliable AI apps — Dmitry Kuchin, Multinear

The standard software development lifecycle and conventional data science metrics are failing to produce reliable AI applications. While creating a proof-of-concept (PoC) that works 50% of the time is straightforward, bridging the gap to a trustworthy, production-level solution is incredibly difficult due to the non-deterministic nature of models. Any change to the code, logic, prompts, models, or underlying data can impact the solution in unexpected ways.

The Flaw in Conventional AI Testing

Many teams mistakenly start with abstract data science metrics like groundedness, factuality, or bias. These metrics are poor indicators of whether a solution is genuinely effective for the end-user. For instance, a customer support bot can be 100% factual and grounded in its source materials but still fail to provide the answer a user needs, leading to frustration and escalation to a human agent. The truly important metric in this case is not factuality, but the “rate of escalation to a human support agent.” The goal is to measure the actual business outcome, not a generic proxy metric.

A New Paradigm: Reverse-Engineering Evals from User Scenarios

The key to building reliable AI is to reverse-engineer your evaluations from real-world scenarios and specific business outcomes. Instead of measuring generic qualities, you must test for very specific criteria derived from the product experience.

This process begins by creating evaluations based on the desired user outcomes. For a customer support bot for a bank, you would start with its FAQ materials.

1. Use an LLM to Generate Evals: A powerful LLM can be used to read source material (e.g., an article on how to reset a password) and generate realistic user questions from various personas.
2. Define Hyper-Specific Criteria: For each question, the LLM also helps define a checklist of specific, non-negotiable points that must be in the answer. For example, a password reset answer might need to mention receiving an SMS code and provide the correct next step if the user no longer has access to that mobile number.
3. Build a Diverse Evaluation Set: This process is repeated to build a large set of evaluations that cover numerous specific scenarios, user personas, and potential questions.

The Evals-First Development Lifecycle

Contrary to traditional development, evaluations are not an afterthought; they are built at the very beginning of the process. The workflow becomes a continuous, iterative loop:

1. Build v1: Develop the first version of your PoC and the first version of your evaluations.
2. Run and Analyze: Execute the evaluations. Crucially, do not just look at average scores. Dive into the details of each failure to understand the root cause—it could be a flaw in the solution or an incorrectly defined test.
3. Improve and Iterate: Make a targeted change to your solution (e.g., tweak a prompt, change the model, adjust the logic).
4. Test for Regressions: Rerun the entire evaluation suite. A fix for one scenario frequently breaks another that was previously working. Without a comprehensive suite of specific tests, these regressions go unnoticed.
5. Continuously Refine: This cycle of improving the solution and the evaluations themselves continues until the application performs reliably across all defined scenarios.

What emerges from this process is a robust benchmark. This baseline provides the confidence to experiment with significant changes—such as swapping in a cheaper model, simplifying the logic, or trying a different RAG strategy—while being certain that core functionality remains intact.

Tailoring Evaluations to the Application

This evaluation strategy is not one-size-fits-all and must be adapted to the specific type of AI application:

• Support Bots: Use an LLM-as-a-judge, checking the output against a list of specific, required criteria for each question.
• Text-to-SQL: The best evaluation is to run the generated query against a mock database with a known schema and data, then verify that the result is exactly what you expect for a given question.
• Classifiers: Evaluations can be a simple match against a predefined rubric.
• Guardrails: These should be treated as another type of evaluation. Create tests for questions that should not be answered or that require a specific, canned response.

By evaluating your application the way a user actually experiences it, you gain a true understanding of its behavior. This detailed, scenario-driven testing process effectively becomes a form of explainable AI, as you know exactly what your solution does and how it performs in the situations that matter most.

• Tags:
• LLM
• Feature
• Testing
• Production AI
• Reliability
• AI Engineering
• Evaluations
• Development Lifecycle