Your agent architecture has a half-life of 6 months — Dan Farrelly, CTO, Inngest

Dan Farrelly, CTO and co-founder of Inngest, argues that agent architectures have a half-life of six months, a reality stemming from the rapid pace of change in models, frameworks, and tool-calling standards. He notes that if you’ve been building agents for over half a year, you’ve likely rewritten components, often because new advancements quickly render existing architectures obsolete. The core question for developers is whether code components survive by accident or by intentional design.

Farrelly proposes a mental model for building resilient agent systems by thinking in three discrete conceptual layers, rather than focusing on specific components:

The Three Conceptual Layers of Agent Architecture

1. Execution Layer (The Brain): This layer handles flow, state, durability, and retries. Farrelly asserts that if designed correctly, this layer can last for years, offering stability amidst rapid change.
2. Context Layer (The Knowledge): This encompasses models, prompts, tools, and memory. This layer is characterized by its extremely short half-life, often changing within weeks or months.
3. Compute Layer (The Hands): This involves sandboxes, runtimes, and automated browsers.

The fundamental problem arises when teams couple these layers together, causing the short half-life of the context layer to leak into and drag down the entire system, creating technical debt. Farrelly’s core thesis is to “think in layers. Decouple them” to enable flexibility and longevity.

The Critical Role of the Execution Layer

Farrelly emphasizes that the execution layer is often overlooked but is the key to building adaptable agent systems. He defines it as “the system responsible for running your code reliably, managing how, when, or whether each piece of work completes,” independently of the underlying infrastructure. It manages the full lifecycle of agent operations: planning, model calls, code execution, sub-agent invocation, looping, retries, and coordination. The goal is that you can swap out models, contexts, or sandboxes, but the execution layer should remain stable.

Key characteristics and requirements of a robust execution layer include:

• Resumability and Durability: Agents are handling longer-running tasks. The execution layer must allow the system to pick up after failures (e.g., failed LLM or tool calls) without restarting from the beginning. This requires state to be durable and external, not held in-memory or on disk within ephemeral processes. Without this, complex, leaky abstractions for manual checkpointing or log-based hydration emerge.
• Flexible Invocation Patterns: A capable execution layer must support a wide array of triggers and workflows, including crons, event-based triggers, APIs, human-in-the-loop interactions, and dynamic sub-agents. It must enable synchronous, asynchronous, and delayed invocations. Without this flexibility, the agent’s “harness logic” begins to absorb concerns like queues, workers, polling, and backoff, leading to poor abstractions.
• Comprehensive Observability: Full session traces across the entire run are essential. This includes not just LLM and tool calls, but also database errors, permissions issues, triggers, and performance metrics. Complete traceability from trigger through the full stack is crucial for debugging, improving, and evolving agents.

Sandboxes: Hands, Not Brains

While sandboxes (a component of the compute layer) are essential for agents to execute code, browse, or manipulate files, they are inherently ephemeral and stateless. Farrelly considers using a sandbox for durability, snapshots, or state management an anti-pattern. Instead, he likens the sandbox to “the hands,” with the execution layer serving as “the brain,” providing its context, sequence, and durability.

Adapting to Future Agent Architectures

Emerging agent trends like background agents, dynamic workflows, autonomous loops, and agent factories all share common traits: they are long-running, asynchronous, and delegated. This necessitates a strong, observable execution layer.

• Background agents, unlike request-response chatbots, run for minutes or hours and involve numerous tool calls, guaranteeing failures. Debugging these without proper infrastructure and observability is impossible.
• Loop architectures continuously assess system state against goals, requiring crons, sub-agent delegation, inspectable history, and reliability. Current frameworks often aren’t designed for this.

Farrelly illustrates these concepts with conceptual code examples:

• Health Check Cron: A cron job runs every 30 minutes, pulls system metrics, and uses an LLM to assess health. If unhealthy, it invokes a triage agent.
• Triage Agent: This agent pulls detailed metrics, initiates an investigation (potentially running for minutes), spins up a sandbox, clones code, and analyzes commits. Its long-running nature demands a durable and observable execution layer.
• Reviewer Function: A weekly cron job reviews the history of the triage system’s performance, analyzing execution logs, sub-agent calls, and overall workflow. It requires “execution aware” and “orchestration aware” capabilities to understand what happened and suggest improvements, thereby completing an “improving loop.”

Measuring and Scoring Agent Performance

The execution layer is an historical hub for observability and scoring because user feedback, actions, and session results naturally flow through it. This allows for instrumenting and scoring agents at key points, attaching outcome-based scores (e.g., “did the triage successfully result in an action by the engineering team?”) by correlating events and understanding the full trace information.

Conclusion

To navigate the rapid evolution of AI agents, developers must build their harnesses with an understanding of these architectural layers. By embracing change and focusing on a robust execution layer with the right primitives, everything else can evolve rapidly around it, making future iterations significantly easier. Inngest offers a solution specifically designed as this durable execution layer for AI agents.

• Tags:
• Observability
• AI Infrastructure
• LLM Agents
• Agent Architectures
• Execution Layer
• Sandboxes
• Durable Workflows
• Architectural Decoupling
• Resumability
• Inngest