Long horizon tasks

Aug 01, 2026

Rethinking Environments for Long-Horizon Work — Rayan Garg, Theta Software

Rayan Garg from Theta Software delves into the complexities of defining and evaluating "long horizon" tasks for AI agents. He critiques current metrics and benchmarks, emphasizing the critical role of sophisticated environment design and robust verifiers (judge models) in driving true progress, particularly in "software-failing domains." The discussion highlights issues like task ambiguity, state changes, and the necessity for granular reward signals for effective model training.

Jul 31, 2026

Agents at Scale: Inside MiniMax's Model and the Infrastructure Behind It — Olive Song

Olive Song, RL lead at MiniMax, details the engineering behind MiniMax's open-weight models, focusing on M3's multimodal and agentic capabilities, the necessity of day-zero inference stack readiness, and continuous GPU kernel optimization. She discusses multimodal training challenges, long-horizon task evaluation, and expresses optimism for open models rapidly closing the gap with frontier labs.

Jul 26, 2026

DeepSWE: A Contamination-Resistant Coding Benchmark — James Shi, Datacurve

DeepSWE is a novel benchmark for evaluating AI models on software engineering tasks. It features 113 original, long-horizon problems from real open-source repositories, designed to resist contamination and provide clearer differentiation between models. The benchmark highlights unique failure modes and strengths of models like Claude and GPT, emphasizes realistic prompt design, and uses program-based verifiers focusing on observable behavior.

Jul 22, 2026

Claude for Long-Horizon Tasks — Lance Martin, Anthropic

Lance Martin from Anthropic shares insights into building reliable and secure long-horizon agents with Claude. He details architectural principles like decoupling the 'brain' from the 'hands' for reliability and security, implementing independent verifiers for self-correction, and developing advanced self-learning memory systems akin to human memory's in-band writing and offline 'dreaming' consolidation. The talk concludes with a vision for evolving agent harnesses towards organizational-level, proactive, and multiplayer capabilities.

Jul 07, 2026

SWE-Marathon: Evaluating Coding Agents at Billion-Token Scale - Rishi Desai, Abundant AI

SWE-Marathon introduces a benchmark for long-horizon autonomous software engineering, pushing coding agents from bug fixes to full project ownership. It highlights the critical need for robust, multi-layered verification and anti-cheat mechanisms to prevent reward hacking in tasks spanning hundreds of millions of tokens, revealing that current agents achieve only a 26% success rate.

Sep 03, 2025

Build Hour: Agentic Tool Calling

A deep dive into building agentic systems using OpenAI's latest APIs. The session covers the core concept of 'agentic tool calling' (reasoning + tools), outlines a four-part framework (Agent, Infrastructure, Product, Evaluation) for designing long-horizon tasks, and provides a hands-on demonstration of building a non-blocking task processing system with a real-time progress UI.