A look at how Optiver, a proprietary trading firm, runs an on-prem AI platform where every model, agent, and harness decision has to show results in P&L within hours. Topics include the inference gateway, same-day adoption of frontier models, the meta harness design, and efforts to bring AI spend back under control.
A workshop on building a self-improvement loop for AI agents: tracing production traces, using an always-on agent to file deduplicated issues, turning failures into evals and regression datasets, proving fixes with experiments, and keeping humans at the highest-leverage review points.
Jesse Hall of LiveKit explains how to build voice agents for financial services that feel human rather than robotic. The talk covers why model leaderboards mislead for real-time conversations, why time to first audio and time to first sentence matter more than first byte or first token, and how async tool calls keep a conversation flowing during slow back-end work. It also covers audio-aware turn detection, the roughly 1.5-second latency budget, and benchmarking your own stack with simulated callers.
A hands-on workshop from Arize AI on moving from vibe checks to real LLM evals. It covers what an eval is and how it differs from a unit test, the main eval types, how to read agent traces, how to write concrete pass/fail criteria, and how to build and iterate on an LLM-as-a-judge in Arize's prompt playground.
Aparna Dhinakaran of Arize AI argues that the AI engineer's job is shifting from writing features to building the loop that builds them: evals that review non-deterministic work, telemetry that agents consume, and always-on agents that act on it. She uses Arize's Signal agent as a case study, covering triggers, sandboxes, durable state, model and harness independence, and evaluating the evaluators.
Aaron Linsky of Bridgewater Associates and Nikhil Benesch of turbopuffer explain how permission-aware retrieval over a 100-million-document research vault broke a conventional search setup, and why native filtering over a clustered vector index is needed to keep search fast without sacrificing recall.