Aditya Khandelwal argues that scaling AI agent adoption within engineering teams is a leadership challenge, not an individual contributor problem. He highlights common pitfalls like agent "babysitting" and "slop," and provides a playbook emphasizing progressive disclosure, high-value automation, robust feedback loops, and a critical mindset shift to successfully integrate agents into team workflows.
Ali Khial exposes critical flaws in popular coding benchmarks, revealing how ambiguous instructions, weak verifiers, and model 'reward hacking' create a disconnect between reported performance and real-world utility. He argues that this leads to a "trust gap" where engineers disregard leaderboards. Khial then outlines five principles for building trustworthy, production-grade benchmarks, emphasizing human-authored instructions, holistic grading, economic value, contamination-free design, and informative leaderboards, urging software engineers to contribute to their improvement.
Robert-Jan Huijsman, Founding Engineer at Reboot.dev, outlines a pragmatic approach to developer productivity by tackling friction and non-determinism in both human collaboration and software development. He shares Reboot.dev's strategies, including async workstreams, full-stack typing, unit testing with AI, and foundational engineering principles like retries, durable execution, and strong consistency, to build reliable and efficient systems.
Boris Cherny, creator of Claude Code, discusses the transformative capabilities of Opus 5, highlighting its prompt injection resistance and long-task execution. He delves into Claude Code's empirical development philosophy of "unhobbling" AI by constantly adapting to new model generations, and shares insights on how to build advanced AI products using higher-level tasks, self-verification, and dynamic workflows to orchestrate thousands of agents.
DeepSWE is a novel benchmark for evaluating AI models on software engineering tasks. It features 113 original, long-horizon problems from real open-source repositories, designed to resist contamination and provide clearer differentiation between models. The benchmark highlights unique failure modes and strengths of models like Claude and GPT, emphasizes realistic prompt design, and uses program-based verifiers focusing on observable behavior.
This talk addresses the critical challenge of verification in AI agent development, moving beyond generation to ensure correctness. It highlights the problem of "AI slop" and the "productivity paradox" of AI coding agents, where initial velocity gains are offset by increased technical debt and quality issues. The speaker introduces the AC/DC (Agent-Centric Development Cycle) framework comprising three stages: Guide (providing context and constraints), Verify (zero-trust, multi-layered verification using both algorithmic and agentic methods), and Solve (active code maintenance to control technical debt). This systems-level approach, integrating verification across agentic, CI, and code maintenance loops, significantly reduces issues and transforms AI into a reliable enterprise asset.