Sep 09, 2026

Inside the Race to Measure Frontier Intelligence

Independent model evaluation is crucial for AI progress, as public benchmarks are saturated and models optimize for them. Rayan Krishnan discusses VALS's role in providing continuously evolving, third-party evaluations, highlighting the Llama 4 'disaster' and the need to measure increasingly agentic systems. The conversation covers the complexities of evaluation beyond capability to include cost and latency, the looming financial impact of token spend on enterprises, and the critical role of evaluations in informing AI policy and international coordination for aligning AI with human intent and managing geopolitical risks, especially concerning recursive self-improvement.

Jul 20, 2026

Why Your Agent Disagrees With Itself (And What To Do About It) - Diane Lin, Datadog

This session addresses the critical problem of AI agent inconsistency, particularly in LLMs, which leads to "flip-flops" in high-stakes domains like cybersecurity. It argues that this isn't a model failure but a signal of ambiguity in the "gray zone" near decision boundaries. The proposed solution involves using active learning to identify these ambiguous cases, followed by augmenting agents with semantic memory (explicit policies) and episodic memory (past similar cases) to clarify decisions, improve consistency, and adapt to customer-specific preferences without relying solely on expensive fine-tuning.

Jul 15, 2026

GLM-5.2: The real security risk? Plus: Vibe hunting, the end of CVSS and updates on Lightwell

This podcast explores the implications of open-weight AI models like GLM-5.2 for cybersecurity, CISA's new four-variable vulnerability prioritization model, the rise of AI-assisted 'vibe hunting,' and the commercial launch of IBM and Red Hat's Lightwell for securing open-source software. It highlights the tension between AI capabilities for attackers and defenders, the challenges of rapid vulnerability remediation, and the need for new "trust infrastructures" in the AI era.