Agent observability

Oct 11, 2026

Let Your Agent Cook: Using Skills to Evaluate and Improve Your App — Ankur Duggal, Arize AI

A financial agent produces confident investment advice with fabricated details, redundant tool calls and bad summaries. This talk shows how traces, evaluations, and a coding agent connected through the Arize CLI and skills can investigate those failures, propose fixes, and test them with experiments. It covers why evaluations matter over vibes-based shipping, how judges and costs fit in, and why human review and application-specific evals are still essential.

Oct 11, 2026

Building Self-Learning Loops for Your Agent — Fuad Ali, Arize AI

Fuad Ali demonstrates how a production failure in a shopping agent, a price filter that ignores budget limits, can become a closed improvement loop. Arize Signal and Arize Skills detect and investigate the failure from traces and code, harness-based judges assess outcomes with tool access and external context, and agent experiments validate the proposed fix against real failing traces before it ships.

Oct 10, 2026

From Your Laptop to the Pipeline: Scaling Custom Agents with GitHub Copilot

Jose Palafox, a GitHub Field Copilot Specialist, walks through taking a Copilot agent from a single laptop to an organization-wide, pipeline-hosted system. Topics include how the Copilot harness and CLI work, building custom agents with the agent builder, sharing them through marketplaces, orchestrating cross-repository changes, and running headless agents in GitHub Actions through GitHub Agentic Workflows, with a focus on observability and cost control.

Oct 08, 2026

Giving AI Agents Memory That Learns — Jake Broekhuizen, LangChain

Jake Broekhuizen of LangChain Labs argues that observability alone does not make agents improve. Using a financial services agent whose tone drifted from advisory to pushy, he shows how traces only become useful memory when a filtered lesson is written back as durable context the agent reads on its next run. The talk covers semantic, episodic, and procedural memory, the working versus long-term memory distinction, the read-write cycle, and how LangSmith and Context Hub support capture, analysis, and updates, along with lessons about keeping only signal, caching, and human review of procedural changes.

Sep 30, 2026

Your Agents Are in Solitary Confinement: Why MCP & A2A Aren't Enough — Vlad Luzin, Band

Vlad Luzin, co-founder and CTO of Band, argues that connecting AI agents is a distributed systems problem that current tools such as MCP, A2A and messaging platforms do not solve. He lays out what is required for real agent collaboration, including ordered real-time transport, persistence, runtime binding, agent-first abstractions and governance, then demonstrates Codex, LangGraph and Claude Code agents finding and working with each other under full human-in-the-loop visibility.

Sep 26, 2026

An AI Research Agent That Runs Your Experiments — Tim Sweeney, Weights & Biases

Tim Sweeney, principal engineer at Weights & Biases, demos ARIA, W&B's agent for automated ML research. ARIA launches and monitors GPU training jobs on Karpathy's autoresearch project, surfaces patterns across 200+ experiments, and builds reports and dashboards. The talk then covers how the team built ARIA with Weave observability, LLM judges, YAML-based tasks as unit tests, and nightly evals, and closes with practical tips for productionizing agents.