Predictability Beats Accuracy in Enterprise AI
Anant Bhardwaj, CEO of Instabase, presents a pragmatic guide for building enterprise AI. He argues that AI agents are best used during the 'design-time' to create predictable workflows, rather than for autonomous 'runtime' operations. Bhardwaj also debunks the hype around RAG, highlighting its dependency on data quality, and explains why trust in AI systems stems from predictability, not just accuracy.
Anant Bhardwaj, CEO of Instabase, offers a practical perspective on implementing AI in the enterprise, emphasizing that the current hype around certain technologies needs to be tempered with a focus on system design, predictability, and solving concrete business problems.
Redefining AI Agents and Their Role
An AI agent is a system that, given a goal and a set of tools, has the freedom to devise and execute a plan to achieve an outcome. Unlike Robotic Process Automation (RPA), which robotically follows a pre-recorded, rigid sequence of steps, an agent can re-evaluate its plan at each step and adapt to unstructured situations. This adaptability is why agents are poised to replace RPA, which fails when conditions change even slightly.
The Controversial View: Agents for Design-Time, Not Runtime
A core, and somewhat controversial, argument is that the primary value of AI agents in the enterprise will be during the build and design time, not at runtime.
- Design-Time (Human-AI Collaboration): In this phase, agents can collaborate with humans to brainstorm, design, and refine complex processes. They can identify blind spots, suggest improvements, and help create robust, intelligent workflows. This is analogous to how coding agents help developers write and debug code before it’s deployed.
- Runtime (Predictable AI Workflows): Production environments, especially for operational workflows like lending or hospital patient intake, demand predictability and reliability. You cannot have an autonomous agent randomly altering a critical process. Once a workflow is designed and validated by humans and agents, it should be deployed as a predictable “AI workflow” that executes consistently every time.
“We believe that the whole agentic buzz… is going to be more valuable during build and design time, not during runtime. Runtime phenomena is going to be once human OKs that whatever workflow that we figured out… then you always run those because then you bring a degree of predictability.”
Debunking RAG and the Enterprise Search Fallacy
When it comes to implementing large language models, the common debate is between Retrieval-Augmented Generation (RAG) and fine-tuning. Bhardwaj’s stance is “neither nor.”
- Fine-tuning is not seen as the future for most applications.
- RAG is an overhyped tool whose effectiveness is entirely dependent on the quality of its underlying retrieval system. The biggest problem with RAG is recall. If the retrieval step fails to find the correct information in the source documents, the LLM has no chance of generating the right answer, regardless of its intelligence. This can be worse than hallucination because it creates a false sense of security.
This leads to another controversial take: enterprise search will remain an unsolved problem until data quality is addressed. Unlike the public internet, which has ranking signals (e.g., trusted domains, inbound links) and curated content, enterprise data (like a collection of Google Docs) is messy. There are no reliable signals to rank documents; recency or author seniority are often misleading. Without solving data quality and ranking, enterprise RAG systems will fail.
The Secret to Evaluation and Trust: Predictability Over Accuracy
Building trust in an AI system is not about achieving the highest possible accuracy percentage. Trust comes from predictability.
A system with 90% field-level accuracy on a document with 60 fields might sound good, but simple math (0.9^60) shows this results in less than 2% document-level accuracy. This means nearly every document requires manual review, rendering the automation useless.
A better system is one designed with checks and balances that can guarantee a subset of outcomes. For example, a system that can process a million documents and confidently certify that 60% of them are 100% correct—requiring no human review—is vastly more valuable. The key is designing a system where you can predict its behavior and know when to trust its output and when to escalate to a human.
“Trust comes from predictability that I can predict which 70% this is correct which 30% is not correct… You trust a predictable horse more than an unpredictable horse even if the other horse might be running faster.”
Starting an Enterprise AI Initiative
The biggest barrier to enterprise AI adoption is often organizational inertia—“too many people, too many opinions, too much management.” To cut through this, enterprises should avoid the mistakes of the “big data” era, where technology (like data lakes) was implemented without a clear purpose.
The correct approach is to start with the problem, not the technology or the data. Identify the five or six most significant challenges the organization faces—such as developer productivity, operational efficiency, or contract analysis—and work backward to see how AI can provide a solution.
A Wishlist for Foundation Model Builders
To better serve enterprise needs, foundation model creators should prioritize:
- Predictability: The number one request. Models should produce consistent, reliable outputs for the same inputs.
- Explainability: The ability to audit how a model arrived at an answer, beyond just token probabilities.
- Better Small Models: Closing the performance gap between large, expensive models and smaller, more efficient models that can run on-device, improving cost, speed, and privacy.