Lawrence Jones from Incident.io explains why their team needed AI to debug their complex AI SRE product. He details three powerful patterns: a CLI for agents to manage large evaluation files, serializing debug UIs into downloadable file systems for agent-based tracing, and multi-agent pipelines for fleet-scale failure analysis.
Agent failures are unlike traditional software failures. This workshop provides a practical framework for monitoring production agents, moving beyond evals to real-world observability by using explicit signals (errors, latency) and implicit signals (user frustration, refusals, self-diagnostics) to catch regressions and understand agent behavior.
Sy Brand demystifies the internal workings of native code debuggers, explaining how core features like breakpoints, stepping, stack unwinding, and variable inspection are implemented using operating system primitives, debug information formats like DWARF, and low-level hardware features.
Emmanuel Ameisen of Anthropic's interpretability team explains the inner workings of LLMs, drawing analogies to biology. He covers surprising findings on how models plan, represent concepts across languages, and the mechanistic causes of hallucinations, offering practical advice for developers on evaluation and post-training strategies.
Professor Andreas Zeller discusses his interactive 'Debugging Book,' arguing that systematic, automated debugging is a critical but neglected skill. He explores powerful techniques like delta debugging and automated repair, explaining how developers can build their own tools to make debugging a more plannable and efficient process.
Sy Brand, author of "Building a Debugger," and Tim Misiak explore how implementing a debugger is one of the most effective ways to gain a deep understanding of operating systems, compilers, and hardware. They delve into the unexpected complexities of core features like stack unwinding and code stepping, the challenges posed by legacy APIs like ptrace and debug formats like DWARF, and the future of debugging with advancements in time travel debugging and tools for optimized code.