Jul 26, 2026

DeepSWE: A Contamination-Resistant Coding Benchmark — James Shi, Datacurve

DeepSWE is a novel benchmark for evaluating AI models on software engineering tasks. It features 113 original, long-horizon problems from real open-source repositories, designed to resist contamination and provide clearer differentiation between models. The benchmark highlights unique failure modes and strengths of models like Claude and GPT, emphasizes realistic prompt design, and uses program-based verifiers focusing on observable behavior.

Jul 26, 2026

State of Data — Sean Cai, Independent / State of Data

Sean Cai discusses the evolving landscape of AI data markets, highlighting the shift from raw annotation to high-quality, process-based data. He introduces "Verifier's Law" and its three axes to explain application layer maturity, critiques the shortcomings of current benchmarks, and predicts future AI trends by analyzing data market signals. Cai concludes by envisioning data companies transforming into "neo-labs" that provide enterprise-level reinforcement learning as a service and "Antikythera mechanisms" to manage and monetize real-world data assets.

Jul 26, 2026

Anthropic’s first technical PM on token maxing, the jagged edge, and living in the future

Dianne Penn, Head of Product for Anthropic’s AI Research and Labs teams, discusses Anthropic's journey, the evolving AI landscape, and the changing role of product management. She highlights the importance of adaptability, 'evals as the new PRDs,' and how Anthropic's focus on alignment and safety makes Claude a more effective 'thinking partner' by enabling it to push back on user ideas.

Jul 26, 2026

Why AI Makes the Humanities More Important Than Ever

Jeff Crume explores why humanities are crucial in an AI-driven world. While AI generates sophisticated answers, it lacks human understanding, purpose, and judgment. He argues that STEM fields explain 'how' but not 'why,' making humanities essential for ethical decision-making, interpreting AI outputs, understanding bias, and effective prompt engineering. Ultimately, AI amplifies the need for human critical thinking and judgment.

Jul 25, 2026

Evals-Driven Development for a Mental Health AI Coach — Akele Reed & Dave Revere, SonderMind

SonderMind's AI mental health coach, Sonder, pioneers an eval-driven development approach balancing effectiveness and safety. This involves a clinical feedback loop turning human therapist insights into machine-readable evaluations, an Ethics Engine with modular, LLM-as-a-judge guardrails for evolving clinical guidelines, and a shift from single-prompt agents to a Supervisor/Executor/Evaluator architecture with human oversight to ensure safety and quality in high-stakes mental health conversations. They also open-source clinically reviewed datasets to foster community safety.

Jul 25, 2026

Loop Engineering from First Principles — Kyle Mistele, HumanLayer

Kyle Mistele, in this presentation, argues that fixing the problem of unreviewable 40,000-line pull requests generated by coding agents doesn't lie in better prompts but in better, control theory-inspired loops. He details how to build agentic control loops with distinct sensors, controllers, and actuators, ensuring incremental, verifiable code changes. Using a practical example of RPC API migration, he demonstrates how to implement these loops with tools like ASGrep, integrate them into CI/CD, enable human feedback, and manage workflow to prevent overwhelming reviewers, ultimately leading to more robust and readable agent-generated code.