A discussion on refining patient questions for a study on diabetes, highlighting the contrast between simplified questions for scalable data collection and the complex, nuanced queries from long-term patients. The team explores how to test their AI-driven storytelling system with these specific, real-world scenarios to generate more grounded and relevant health narratives.
Dexter Horthy of Human Layer explains why naive AI coding agents fail in complex software projects and introduces 'Advanced Context Engineering.' He details a spec-first, three-phase workflow (Research, Plan, Implement) designed to manage context intentionally, keeping utilization below 40% to maximize model performance. This approach uses subagents and frequent compaction to turn AI from a prototyping tool into a production-ready system for large, brownfield codebases.
Sam Bhagwat of Mastra details their process for optimizing AI agent memory using the Long Mem Eval benchmark. He breaks down memory into subtasks like temporal reasoning and knowledge updates, and shares how targeted improvements—such as tailored templates, targeted data updates, and structured message formatting—led to state-of-the-art performance, emphasizing the importance of iterative evaluation.
Jeff Huber of Chroma argues that building reliable AI systems hinges on 'Context Engineering'—the deliberate curation of information within the context window. He challenges the efficacy of long-context models, presenting a 'Gather and Glean' framework to maximize recall and precision, and discusses specific challenges and techniques for AI agents, such as intelligent compaction.
Jake Heller, founder of Casetext, shares a pragmatic framework for turning powerful large language models like GPT-4 into reliable, professional-grade products. He details a rigorous, evaluation-driven approach to prompt and context engineering, emphasizing iterative testing, the critical role of high-quality context, and advanced techniques like reinforcement fine-tuning and strategic model selection.
Gal Peretz introduces CodeAct, a paradigm where LLMs generate and execute Python code for tool interaction, offering a more flexible and powerful alternative to traditional JSON-based function calling for building complex, production-ready AI agents.