Memory management

No Memory, No Harness: Why the Database Is the Last Line of Defense — Kay Malcolm, Oracle

No Memory, No Harness: Why the Database Is the Last Line of Defense — Kay Malcolm, Oracle

Kay Malcolm discusses how AI, while boosting individual performance, often hinders overall team productivity due to fragmented context. She introduces the concept of an "agent harness" with "memory" as its central nervous system, vital for sharing context. Drawing on personal experiences with database sprawl, she highlights the inefficiencies of storing diverse memory types across multiple data stores. Malcolm then presents Oracle's AI database (26AI) as a unified solution, capable of natively managing all memory types in a single platform, thereby enhancing team collaboration and solving the context problem for enterprise AI agents.

Deep dive on LLM Inference at Scale — Harshul Jain, Audible & Tanmay Sah, Independent AI Researcher

Deep dive on LLM Inference at Scale — Harshul Jain, Audible & Tanmay Sah, Independent AI Researcher

A workshop by Harshul Jain and Tanmay Sah breaks down LLM inference optimization from first principles, addressing memory, TTFT, and throughput issues. It covers foundational concepts like KV cache mathematics and the prefill/decode phases, then dives into model-side optimizations (quantization, attention variants like GQA, FlashAttention) and serving-side techniques (Paged Attention, continuous batching, prefix caching, KV quantization). The session concludes with benchmarks of popular inference engines like vLLM and SGLang, highlighting their strengths for different workloads, and outlines future research directions.

Agents Are Where Microservices Were in 2015 — Roberto Milev & Uday Kanagala, Navan

Agents Are Where Microservices Were in 2015 — Roberto Milev & Uday Kanagala, Navan

Roberto Milev and Uday Kanagala from Navan discuss the challenges and emerging best practices in building and operating AI agents, drawing parallels with microservices. They highlight the shift to stateful runtimes, innovative memory and context management using 'skills,' and crucial solutions for observability and testing non-deterministic systems. A central theme is the critical need for robust authorization and guardrails, given the blurring line of agentic action, and a preference for mastering single-agent loops before scaling to complex orchestrations. They conclude by outlining the solved and unsolved problems, with cost and debugging remaining significant hurdles.

Claude for Long-Horizon Tasks — Lance Martin, Anthropic

Claude for Long-Horizon Tasks — Lance Martin, Anthropic

Lance Martin from Anthropic shares insights into building reliable and secure long-horizon agents with Claude. He details architectural principles like decoupling the 'brain' from the 'hands' for reliability and security, implementing independent verifiers for self-correction, and developing advanced self-learning memory systems akin to human memory's in-band writing and offline 'dreaming' consolidation. The talk concludes with a vision for evolving agent harnesses towards organizational-level, proactive, and multiplayer capabilities.

Turn 10,994 Notes Into Memory - Paul Iusztin, Decoding AI & Louis-François Bouchard, Towards AI

Turn 10,994 Notes Into Memory - Paul Iusztin, Decoding AI & Louis-François Bouchard, Towards AI

Paul Iusztin and Louis-Francois Bouchard unveil their AI Research OS, a powerful system that transforms notes, documents, videos, and code repositories into a dynamic, personalized memory for AI agents. They detail a three-layered architecture (raw files, index, wiki) that efficiently manages context and enables continuous learning and evolution, addressing the limitations of static research and generic AI tools.

Hierarchical Memory: Context Management in Agents — Sally-Ann Delucia

Hierarchical Memory: Context Management in Agents — Sally-Ann Delucia

The Arize team shares lessons from building their AI agent, Alyx, which analyzes its own trace data. They detail their journey from failed attempts like naive truncation and summarization to a successful strategy combining head/tail preservation with a retrievable memory store and using sub-agents to manage context complexity.