The 6 Pillars of an Agentic Harness for Production — Varun Krovvidi, Resolve AI
Varun Krovvidi of Resolve AI explains why AI excels at code generation but struggles with the complex, multi-domain problem of running and fixing software in production. He outlines five common failure modes for AI agents in production (anchoring bias, context issues, coherent vs. causal answers, guardrails, lack of learning) and introduces Resolve AI's six-pillar architecture: model orchestration, context engineering, causal reasoning, governed actions, learning systems, and multi-level evaluations. A demo illustrates how Resolve AI diagnoses a Grafana alert to its root cause, ruling out coincidental events and maintaining its causal reasoning.
Varun Krovvidi, Product Marketing Lead at Resolve AI, argues that while AI has revolutionized code generation due to its self-documenting, modular, and single-domain nature, its application to running and fixing software in production presents a significantly harder challenge. Approximately 70% of engineering time is dedicated to operational tasks like on-call, incident resolution, and daily maintenance, which are complex, multi-domain, and multi-system problems requiring collaboration across numerous teams and tools.
Why Production Differs from Code Generation for AI
Krovvidi highlights fundamental differences that make production AI challenging:
- Multi-domain and Multi-tool: Unlike code, production systems span code, infrastructure, telemetry, and various specialized tools.
- Multiplayer Problem: Resolving production issues requires coordination across many specialized teams (SREs, platform engineers, backend engineers, etc.).
- Single Correct Answer: Production tasks demand a precise, correct answer and actionable fix, not merely coherent generation. This “last mile” is the hardest.
Five Cracks as Production Agents Scale
Pointing a general-purpose model at production data leads to familiar failure modes:
- Anchoring Bias and Model Treadmill: Models tend to provide coherent answers, not necessarily causal ones. Continuous evaluation and adaptation to new models are required, mapping the best model to specific tasks (e.g., image reasoning vs. deterministic steps).
- Context Window Challenges: Too much context leads to over-exploration and irrelevant hypotheses, while too little results in underexploration, missing crucial information in an environment with infinite telemetry.
- Coherent vs. Causal Reasoning: Models are designed to please and provide coherent responses, but production incidents demand a causal chain of evidence to identify the true root cause, not just a plausible story.
- Lack of Guardrails: Without explicit guardrails, AI might determine destructive actions (e.g., deleting a file system) as the “cleanest fix.” Least privilege access is crucial.
- Missing Learning Loops: Production is a continuous learning environment. Agents need to learn from past investigations, user interactions, and temporal discussions to avoid starting from scratch for every incident.
The Six Pillars of an Agentic Harness for Production AI
Resolve AI addresses these challenges with a six-pillar architecture designed for domain-specific AI:
- Model Orchestration: An engine to keep pace with evolving models and match the most suitable model (e.g., Gemini for images, OpenAI for deterministic tasks, Claude for open-ended investigations) to each specific sub-task within a workflow.
- Context Engineering: Focuses on providing the precise amount of context needed. This often combines techniques like graph RAG for initial investigations with precise tool calls (for logs, metrics, dashboards) to avoid token waste and over/under-exploration.
- Causal Reasoning: Built into the system to always establish a causal chain of evidence for the root cause. If such a chain cannot be established, the system reports low confidence and points the user toward further investigation.
- Governed Actions: Defines specific guardrails and access levels (read/write permissions, conditional writes) for AI operations based on organizational policies and industry regulations.
- Learning Systems: Every interaction with the AI system is an opportunity for learning. The system learns from investigations and user feedback (positive/negative reinforcement, steering the investigation).
- Evals at Five Levels: A critical component from start to finish, evaluating the system’s performance based on:
- Positive and negative reinforcement from users.
- Tracing the AI’s path to a solution.
- Scoring its performance against an ideal human engineer.
- AI’s self-calibration and confidence level in its findings.
Demo: From Alert to Root Cause
Krovvidi demonstrated Resolve AI’s platform, showing how it processes a Grafana alert as a prompt. Investigator agents gather information from metrics, traces, logs, change events, code, infrastructure, and knowledge bases to correlate data and provide a root cause. The demo showcased:
- Identifying a log skill experiencing a high failure rate due to stale integrations.
- Explicitly ruling out other plausible theories, such as a coincidental GCP outage or traffic spikes, preventing misdirection.
- The agent’s ability to hold its ground when challenged, only basing its answers on the established causal chain of evidence.
- Facilitating a multiplayer experience by allowing team members to collaborate and interrogate the AI’s findings directly from Slack.