Agents Are Where Microservices Were in 2015 — Roberto Milev & Uday Kanagala, Navan

Roberto Milev and Uday Kanagala from Navan discuss the challenges and emerging best practices in building and operating AI agents, drawing parallels with microservices. They highlight the shift to stateful runtimes, innovative memory and context management using 'skills,' and crucial solutions for observability and testing non-deterministic systems. A central theme is the critical need for robust authorization and guardrails, given the blurring line of agentic action, and a preference for mastering single-agent loops before scaling to complex orchestrations. They conclude by outlining the solved and unsolved problems, with cost and debugging remaining significant hurdles.

August 29, 2026 5 min read

Agents Are Where Microservices Were in 2015 — Roberto Milev & Uday Kanagala, Navan — source: https://www.youtube.com/watch?v=32nrHU6zHU8

The speakers, Roberto Milev (Chief Architect at Navan) and Uday Kanagala (architecture team), share Navan’s insights on operating AI agents, drawing parallels with the evolution of microservices. They argue that just as it was ill-advised to jump to microservices without mastering a well-structured monolith, it’s prudent to perfect a single agentic loop before attempting complex multi-agent orchestrations.

An emerging reference architecture for reliable agentic flows in production includes layers such as runtime, memory, context management, operational concerns, and orchestration.

Runtime and Stateful Agents

A significant shift from stateless microservices, AI agents are inherently stateful, requiring persistent sessions, isolation, and a distinct lifecycle. While cloud providers (e.g., AWS Agent Core Runtime) offer solutions, Navan has developed custom components like session persistence and rehydration. These runtimes are generally framework-agnostic but often favor native options.

Memory Management

Initial reliance on Retrieval-Augmented Generation (RAG) was driven by the necessity to manage context window limitations. The industry has converged on a pipeline for memory—ingestion, extraction, consolidation, and retrieval. Memory evolves from short-term conversational to long-term semantic storage, including “episodic memories” of successful and failed instances. Navan leverages AWS Agent Core Memory, adapted for its specific use cases.

Context Management and Skills as Units of Work

Despite growing context windows, managing context remains critical. Too much context can lead to agents losing focus. Navan’s approach is to define “skills as a unit of context.” A skill encompasses domain-specific instructions and setup, alongside the actual tool execution. These skills are designed to be pluggable, independently testable, and reusable. Context is dynamically composed from relevant skills, utilizing “progressive disclosure” to start with a limited scope and expand as needed with metadata.

Observability and Debugging Agentic Systems

Traditional logging, effective for microservices, falls short with agents due to the sheer volume of “thinking” output. To address this, Navan implements hooks that intercept every tool call and decision point (pre/post-tool call, pre/post-decision). These interception points allow for blocking operations, logging, and emitting metrics. Critical signals embedded in auto-traces (spans) include the agent’s current goal, reasoning, belief status, and tool calls. A key output is a confidence score associated with each decision, indicating whether an answer is inferred or based on multiple paths. Inferred answers can be routed to a human-in-the-loop for review and agent refinement, providing crucial operational confidence.

Testing Non-Deterministic Agents

Testing agents presents a challenge due to their non-deterministic nature, a stark contrast to predictable, deterministic code. Unlike programmed algorithms with fixed sequences, agents generate their own steps dynamically. Consequently, asserting fixed outputs is ineffective. Navan adopts a research-backed approach of scoring trajectories. Instead of evaluating specific outputs, the focus is on assessing the “efficiency or the completeness” of the agent’s journey from its starting point to its goal, irrespective of the exact intermediate steps. “Inferred signals” derived from confidence scores help classify regressions.

Authorization and Guardrails

A critical concern in enterprise AI is authorization, particularly the blurring line of accountability when agents act “on behalf of users.” For example, if an agent books a flight when the fare drops below a certain price, the question arises: “Who made that purchase?” The traditional model of a user or service principal is insufficient. Navan addresses this with fine-grained authorization decisions and guardrails implemented before and after every tool call. This policy layer checks and blocks actions to ensure governance and prevent sensitive information leaks.

Agent Orchestration: Single Master vs. Multi-Agent

Navan advocates for mastering a single agentic loop before scaling to multi-agent systems. Their strategy employs a single master agent that progressively loads specific skills (sub-agents) into its context as needed. This allows for focused, decisive navigation through use cases. However, they acknowledge emerging patterns like Agent-to-Agent (A2A) communication protocols, which facilitate structured interactions between agents, potentially serving as boundaries between distinct organizational teams.

The Maturity Landscape: Solved vs. Unsolved Challenges

Roberto and Uday conclude by categorizing the maturity of various aspects:

Solved/Mature Areas:

Runtime: Considered “pretty much solved” with advanced orchestration capabilities and scalable LLM deployments.
Memory: Gaining maturity with improvements in frontier LLMs and cloud provider offerings.
Tool Calling: The Multi-Modal Communication Protocol (MCP) has emerged as a de-facto standard, widely supported and evolving towards statelessness, simplifying service and tool invocation for agents.

Unsolved/Challenging Areas:

Observability: While OTEL (OpenTelemetry) is a push, its efficacy for agentic calls is still debated, though it can be adapted.
Testing: Remains “very hard,” but solutions are emerging to deliver quality experiences despite inherent unreliability.
Orchestration: While patterns are developing, the advice is to avoid over-engineering.
Cost: This is “genuinely unsolved” and difficult to predict or manage. The speakers note that vendors often incentivize higher token usage. Solutions might involve using cheaper models for specific tasks or implementing fallbacks.
Replay and Debugging: Debugging agent behavior is complex, but future solutions might involve agents assisting in their own debugging.
Standards: Emerging standards like OTEL and A2A are still nascent and often vendor-driven, but are expected to mature over time.

The presentation concludes with the understanding that while progress is being made, there’s still significant work for the community to define and build the necessary infrastructure for robust AI agent systems.