Code generation

DeepSWE: A Contamination-Resistant Coding Benchmark — James Shi, Datacurve

DeepSWE: A Contamination-Resistant Coding Benchmark — James Shi, Datacurve

DeepSWE is a novel benchmark for evaluating AI models on software engineering tasks. It features 113 original, long-horizon problems from real open-source repositories, designed to resist contamination and provide clearer differentiation between models. The benchmark highlights unique failure modes and strengths of models like Claude and GPT, emphasizes realistic prompt design, and uses program-based verifiers focusing on observable behavior.

In the Land of AI Agents, the Verifiers Are King — Tariq Shaukat, Sonar

In the Land of AI Agents, the Verifiers Are King — Tariq Shaukat, Sonar

This talk addresses the critical challenge of verification in AI agent development, moving beyond generation to ensure correctness. It highlights the problem of "AI slop" and the "productivity paradox" of AI coding agents, where initial velocity gains are offset by increased technical debt and quality issues. The speaker introduces the AC/DC (Agent-Centric Development Cycle) framework comprising three stages: Guide (providing context and constraints), Verify (zero-trust, multi-layered verification using both algorithmic and agentic methods), and Solve (active code maintenance to control technical debt). This systems-level approach, integrating verification across agentic, CI, and code maintenance loops, significantly reduces issues and transforms AI into a reliable enterprise asset.

Sandboxing, Agent Harnesses, and Agent Teamwork

Sandboxing, Agent Harnesses, and Agent Teamwork

Shahram Anver, CEO of Cleric, details how AI agents for SRE are evolving beyond fast triage to continuous learning and operational memory. He discusses Cleric's architectural shifts, from complex early designs to simpler, sandboxed query agents, and the unique challenges SRE agents face in diverse production environments. A core focus is on human-agent interaction, redefining roles as managers overseeing agents, and how agents learn from unstructured data like Slack to build robust, actionable knowledge for autonomous, self-healing infrastructure. The discussion also touches on the future of software, differentiating between durable systems and rapidly developed "skills" or "vibe-coded" solutions.

Content Is Code - Matt Palmer, Conductor

Content Is Code - Matt Palmer, Conductor

Matt Palmer discusses how code, empowered by AI, is becoming the fastest and most efficient medium for technical content creation, emphasizing that while code is cheap, structure and conscientiousness are now the most valuable assets for producing high-quality output. He predicts the rise of "content engineers" who leverage declarative pipelines to automate technical communication.

Research to Reality with Google DeepMind — Benoit Schillings, Google DeepMind, VP of Technology

Research to Reality with Google DeepMind — Benoit Schillings, Google DeepMind, VP of Technology

Benoit Schillings, VP of Technology at Google DeepMind, explores the evolution of AI's role in software development, highlighting the transition from human-limited coding to an AI frontier where syntax generation is solved. He delves into the power of self-play for model training, the shifting economics of software engineering, and the imperative for active guardrails. Schillings also discusses the need for inductive architecture, advanced model planning, multimodal reasoning (as seen in Gemini), and the potential for AI to drive scientific breakthroughs in fields like chemistry and biology by uncovering patterns imperceptible to human bias.

Why a Nation Can't Outsource Its Frontier AI - Alistair Pullen (Cosine AI)

Why a Nation Can't Outsource Its Frontier AI - Alistair Pullen (Cosine AI)

Alistair Pullen, CEO of Cosine, discusses the UK's sovereign AI initiative, born from US export controls. He outlines Cosine's unique economic model, competing with "millions" against "billions" by licensing models instead of hosting inference. Pullen delves into why open models lag frontier systems, emphasizing active parameters and post-training data. He explains Cosine's innovative approach to "slop" through process-based RL and credit attribution, advocating for runtime proof in code review. The conversation covers their hierarchical "Swarm" sub-agent system, the challenges of memory, and advanced synthetic data generation, concluding on the geopolitical impact of export controls as an unexpected catalyst for UK AI.