Oct 22, 2026

How to Test AI Agents When No Benchmark Exists — Nathanael Weill, Air Canada

Nathanael Weill of Air Canada explains how to evaluate AI agents when no public benchmark exists and expert time is scarce: start from a known answer, generate the input from it, and check whether the agent recovers it. He covers lessons from a telecom RAG project that moved from one expert-graded test a week to dozens a day, and five ways to build test sets for a supplier contract review agent, along with why code-based checks are preferred over LLM judges.

Oct 21, 2026

The AI Content System Behind 200M Views — Sarah Chieng, Cerebras & Cova Lee, OpenAI

Sarah Chieng (Cerebras) and Cova Lee (OpenAI) walk through an agent-powered content system spanning ideation, scripting, recording, editing, repurposing, and publishing, emphasizing swipe files, voice-driven scripts, design systems, API-based editing, and weekly process reviews as ways to free creators to focus on craft over generic AI output.

Oct 21, 2026

Does Code Quality Still Matter? It Depends on This — Max Kanat-Alexander, Capital One

Max Kanat-Alexander reframes the familiar "it depends" answer around two factors that govern all software work: understanding a system and safely changing it. He explains why code quality matters for durable software but not for a new, rapidly growing category of disposable software created by AI, and shows how to make systems more disposable and make rigor cheap through standardization, inward-flow tooling, easier testing, and maximum strictness.

Oct 21, 2026

One Day, $1,000 in Tokens: An AI War Room for Legacy Code — Ford Prior, GEICO

Ford Prior, a senior staff engineer and tech evangelist in GEICO's claims technology group, describes an AI-native multiplayer war room built to unblock major, outer-loop software changes. The talk covers a one-day, $1,000 token experiment across rentals, damages, and salvage legacy domains, the FleetKit markdown rulebook that steers the agent fleet, the worksheets and design artifacts it produced, and the cost, model-selection, and tooling lessons that shape what comes next.

Oct 21, 2026

AI Cut Our Research Time From 7 Days to 15 Minutes — Gareth Yoder, Vanguard

Gareth Yoder of Vanguard describes modernizing a cash ledger handling up to 30 million calls per day by running data, producer, and consumer migrations in parallel. Key themes include exposing legacy code to AI through an MCP server, reducing a seven-day research cycle to fifteen minutes, reimagining rather than lifting and shifting, and maintaining observability throughout the migration.

Oct 21, 2026

Is a Faster Model Actually Faster? Test the Whole Job — Morgan Suo, MiniMax

Morgan Suo of MiniMax tests the changes inside M3.1 Flash on a realistic analyst research task about circuit-board suppliers and AI-driven cash flow. She examines sparse attention, 4-bit KV caches, low-precision MoE experts, speculative decoding, serving concurrency, and reasoning budgets, showing how each saves compute while potentially costing accuracy or latency. She closes with a framework for judging a model switch by verifiable output, the full cost of a usable report, and repeatable results.