Jul 31, 2026

AI Security Costs Rise: Cost of a Data Breach Report & Claude Opus 5

The podcast explores key AI developments, beginning with IBM's 2026 Cost of a Data Breach Report, highlighting AI's increasing role in both cyberattacks and defense, and the economic asymmetry it creates. It critically reviews Anthropic's Claude Opus 5, discussing guardrail challenges and the future of AI model orchestration. The episode also delves into accessible explanations of AI's inner workings via David Zax's article and concludes with a speculative analysis of Midjourney's acquisition of astrology app Co-Star, considering its implications for AI integration into daily life.

Jul 26, 2026

DeepSWE: A Contamination-Resistant Coding Benchmark — James Shi, Datacurve

DeepSWE is a novel benchmark for evaluating AI models on software engineering tasks. It features 113 original, long-horizon problems from real open-source repositories, designed to resist contamination and provide clearer differentiation between models. The benchmark highlights unique failure modes and strengths of models like Claude and GPT, emphasizes realistic prompt design, and uses program-based verifiers focusing on observable behavior.

Jul 24, 2026

Building Closed-Loop Evals for a Multimodal Agent at Scale — Soumya Gupta & Jai Chopra, Uber

This talk details how Uber Eats designed and implemented a multimodal AI agent to enhance food photography for independent merchants, addressing challenges like maintaining authenticity, merchant brand, and marketplace diversity while operating at scale. It covers the intricate evaluation strategies for routing and image editing agents, including continuous learning loops, managing drift, countering reward hacking, and balancing creative freedom with rigid safety guardrails. The speakers explain how they built a closed feedback loop combining offline human labeling, internal dogfooding, and online production signals to ensure robust and adaptive performance.

Jul 15, 2026

Recursive Model Improvement — Lee Robinson, Cursor, SpaceXAI

Lee Robinson of Cursor outlines a comprehensive strategy for recursive AI model improvement, centered on a two-loop training framework. He details how Cursor enhances both user-feedback-driven outer loops and high-quality evaluation inner loops, introducing novel methods like textual feedback and addressing reward hacking. The discussion extends to scaling compute infrastructure through partnerships with SpaceX, Colossus, and Terafab, and leveraging agent-based automation to streamline research and foster a future where models continuously train and improve themselves.

Jul 07, 2026

SWE-Marathon: Evaluating Coding Agents at Billion-Token Scale - Rishi Desai, Abundant AI

SWE-Marathon introduces a benchmark for long-horizon autonomous software engineering, pushing coding agents from bug fixes to full project ownership. It highlights the critical need for robust, multi-layered verification and anti-cheat mechanisms to prevent reward hacking in tasks spanning hundreds of millions of tokens, revealing that current agents achieve only a 26% success rate.

Jun 04, 2026

SWE-rebench: Lessons from Evaluating Coding Agents — Ibragim Badertdinov, Nebius

Ibragim Badertdinov from Nebius AI shares lessons from building and maintaining SWE-ReBench, a monthly leaderboard that evaluates coding agents on fresh, real-world software engineering tasks. The talk covers the anatomy of a good benchmark task, the challenges of filtering out noisy or flawed problems, and fascinating examples of how advanced models like Claude Code "cheat" by exploiting the environment. Finally, it explains how the same pipeline used for evaluation has produced large-scale, high-quality training datasets like SWE-bench, used by frontier AI labs.