Why Your Agent Disagrees With Itself (And What To Do About It) - Diane Lin, Datadog
This session addresses the critical problem of AI agent inconsistency, particularly in LLMs, which leads to "flip-flops" in high-stakes domains like cybersecurity. It argues that this isn't a model failure but a signal of ambiguity in the "gray zone" near decision boundaries. The proposed solution involves using active learning to identify these ambiguous cases, followed by augmenting agents with semantic memory (explicit policies) and episodic memory (past similar cases) to clarify decisions, improve consistency, and adapt to customer-specific preferences without relying solely on expensive fine-tuning.
