Two Roads to Durable Agents: Replay vs. Snapshot — Eric Allam, Trigger.dev
This talk explores the architectural shift required to build durable, long-running AI agents. It argues against traditional replay-based durability, proposing a two-part solution: treating LLM context as a durable, append-only log and managing the compute state (memory, files, processes) with OS-level snapshot and restore, implemented efficiently using Firecracker microVMs.
For the last 30 years, backend infrastructure has been dominated by the “shared-nothing” architecture, where the compute layer remains stateless and relies on a database for state. This model, from CGI and the LAMP stack to modern serverless functions, is being fundamentally challenged by the rise of long-running, stateful AI agents. An agent isn’t a short-lived transaction; it’s a long-running session that needs to be durable across user interactions, code versions, and potential failures.
The Limitations of Replay-Based Durability
The current prevailing method for achieving durability is the “replay model,” borrowed from workflow engines. In this paradigm, every significant operation—each LLM call, each tool execution—is wrapped as a step and recorded in a journal. On recovery or resumption, the entire function re-executes, replaying the journal to skip completed steps and resume from the point of failure.
This approach works for well-defined, multi-step workflows but breaks down for agents for several key reasons:
- Unbounded Journal Growth: As an agent interacts over hours or days, the replay log grows continuously, eventually hitting the fundamental limits of the system in either size or number of entries.
- Rigid Structure: It forces code into a rigid structure where everything outside of a “step” must be deterministic, constraining development.
- Session vs. Transaction: The replay model is designed for finite transactions, whereas an agent is an ongoing session that should persist for as long as the user requires.
A New Model: Separating Context and Execution
A more robust approach is to separate the problem of agent durability into two distinct halves:
-
Context Durability: This refers to the history of all interactions with the LLM—system messages, user inputs, tool calls, tool results, and assistant responses. This is effectively an append-only log. This log is inherently durable and can be reliably stored using existing technologies like a database, object storage, or a distributed file system. Making the context log durable provides resilience against machine crashes and allows for safe upgrades to the agent’s code, as the core interaction history is preserved independently.
-
Execution Durability: This encompasses the state of the compute layer itself: files on the filesystem (e.g., a cloned git repo), data loaded into memory, and running subprocesses (e.g., a dev server). This state is ephemeral and cannot be captured in a simple log. The solution here is not a better log, but OS-level snapshot and restore. This allows the entire machine state to be frozen, saved to disk, and then restored almost instantly when the agent needs to resume.
This combined approach provides durability across user turns (snapshotting while waiting for input), across failures (recovering from the context log or a previous snapshot), and across code versions (re-hydrating a new environment with the context log).
Implementation with Firecracker MicroVMs
This concept of “checkpoint and restore” is not new, dating back to IBM mainframes in the 1960s. A modern implementation at Trigger.dev leverages Firecracker microVMs to snapshot the entire virtual machine.
A naive snapshot of a VM can be prohibitively large (e.g., a 512MB snapshot for a 512MB VM). The key to making this practical is optimization:
- Seekable Compression: Compressing the memory state allows for significantly smaller snapshots. By using a seekable format, memory pages can be decompressed on-demand during restore, rather than all at once.
- Snapshot Layering: Additional techniques further reduce the size of incremental snapshots.
Through these methods, a full VM snapshot can be reduced to as little as 14 megabytes compressed. This enables sub-second snapshot times and restore times of a few hundred milliseconds, making the process fast enough for interactive use.
To facilitate this, a new open-source tool called FC Run is being released. It provides a Docker-like CLI for running and managing containers in Firecracker VMs, with built-in, high-performance snapshot and restore capabilities. This tooling is capable of starting ~15,000 VMs per minute, demonstrating its efficiency and readiness for production workloads. This marks a necessary evolution from stateless to stateful compute to power the next generation of AI agents.