The 5 Levels of Self-Driving Production — Eric Schwartz, Traversal
Eric Schwartz, a product manager at Traversal, argues that AI-generated code has moved engineering time from building to troubleshooting, and that observability dashboards show what broke but not why. He frames root cause analysis as a causal problem, lays out five levels of self-driving production, walks through alert intelligence at PepsiCo and incident response at American Express, and offers five questions for evaluating an AI SRE.
Coding agents shifted the bottleneck from building to troubleshooting
Software engineering breaks into three buckets: system design, development, and troubleshooting. Coding agents have compressed development dramatically, so a team that once spent most of its effort writing code can now produce far more of it. The expectation was that this freed time would go to architecture and design. What enterprises actually report is the opposite: more code, less understanding of what is being pushed to production, and more time spent on troubleshooting. The speaker cites industry figures of roughly $400 billion a year in enterprise losses, 40% of executives reporting the problem, and engineers losing seven-plus hours a week to on-call troubleshooting.
Why observability tools stop short
Observability platforms such as Datadog, Elastic, Splunk, and ServiceNow excel at telling teams that something is broken and surfacing correlated signals. They do not explain why it broke or what to do next. Adding dashboards does not close that gap as complexity grows. The speaker notes that Google’s SRE handbook has addressed this limitation directly, and that Anthropic has acknowledged LLMs struggle to identify root causes.
Root cause analysis is a causal problem
The difficulty is structural. A failing checkout API may require five to ten hops across dozens of services and petabytes of data before reaching the true cause, such as an expired TLS certificate. In a small environment a single senior SRE can hold the full stack in their head. At Fortune 50 or Fortune 100 scale, no one does, so incidents turn into war rooms with dozens of engineers, each holding a partial slice of context. Humans paired with coding assistants face the same fragmentation. The core thesis is that root cause analysis is a causal inference problem, not an observability problem. Traversal’s founders, three from academia and one from quantitative finance, have spent years on causal machine learning, the study of how to establish cause and effect.
Five levels of self-driving production
The speaker frames autonomy in levels, borrowing from self-driving cars. The target is a closed loop in which the system detects an incident, identifies its root cause, proposes a fix, verifies the fix, and closes the loop without paging anyone.
- Level 0: Fully manual, with people pulled into Slack war rooms and Zoom calls for hours of debugging.
- Level 1: Rule-based automations and team-built agent loops that work for structured, familiar alerts but break down on novel incidents that lack a runbook.
- Level 2 and 3: Greater automation, including homegrown agents that handle specific services or alert sets well.
- Level 4: Autonomy across an entire environment, with hundreds of services, repositories, and log indexes. The speaker identifies this as the major leap, since building such a system in-house is very difficult.
- Level 5: Full diagnosis across a large enterprise production environment, plus generating and verifying fixes.
Use cases in practice
Traversal describes its product surface through several use cases: alert intelligence for triaging alert fatigue, incident root cause analysis, self-healing that proposes and applies mitigations, production support for ad hoc questions, and code resilience checks before changes ship. The speaker reports that the platform handles trillions of logs, trillions of spans, and tens of billions of metrics and events, and reaches 80%+ root cause accuracy on high-severity incidents at that scale.
For PepsiCo’s supply chain team, the problem was alert overload. Engineers received thousands of alerts weekly, with individual backlogs reaching 700. Traversal pre-investigates alerts and surfaces only those worth attention, recommends alert rule changes to reduce noise, dismisses non-urgent items, and files tickets for deferrable technical debt.
At American Express, a typical incident had five to ten teams paged and 20 to 50 engineers involved, with resolution taking an hour or more and sometimes days. Traversal is now the first responder to every incident. It is dispatched within three minutes of the bridge being declared, posts a detailed root cause analysis to the incident Slack channel, and can update ServiceNow tickets. Teams are often paged only to verify the findings, saving dozens of engineers from disruption, particularly during overnight incidents.
Five questions for evaluating an AI SRE
The speaker proposes five tests for any AI site reliability engineering product:
- Coverage: Can it see all production data, or only a slice with gaps?
- Scale-efficient search: Can it search petabytes and hundreds of billions of logs without ballooning cost or degrading observability infrastructure?
- Relationship mapping: Can it model how entities in the data relate to one another? Ingesting data without understanding its structure is of little use.
- Autonomous improvement: Does it get smarter over time without requiring engineers to maintain markdown runbooks or embedded forward-deployed engineers to keep system knowledge current?
- Multi-hop reasoning: Can it find a non-obvious root cause several hops from the symptom within minutes? The speaker says on-call patience runs out after about five minutes, after which engineers revert to their own habits.
Traversal’s architecture
The stack has three layers. At the base, data is ingested and integrated from connected sources without driving up cost. In the middle, a production world model maps relationships among ingested entities. At the top, those insights are packaged into use cases and user experiences. The two core components are the production world model and what Traversal calls its causal search engine, the mechanism that gives agents a structured way to query the mapped data. Together, the speaker argues, these are how Fortune 500 enterprises can reach level five autonomy.