Context Engineering Our Way to Long-Horizon Agents: LangChain’s Harrison Chase

The development of AI agents has reached an inflection point where long-horizon agents—those capable of operating autonomously for extended periods—are becoming increasingly viable. This progress isn’t just due to more powerful models, but a fundamental shift in how we build and manage them.

The Rise of Long-Horizon Agents

The core idea of an agent has always been an LLM running in a loop, making its own decisions. Early attempts like Auto-GPT captured the public’s imagination but were limited by model capability and the surrounding infrastructure. Today, we are seeing a resurgence, primarily because of two factors:

1. Improved Models: Foundational models have become significantly better at reasoning.
2. Smarter Infrastructure: The community has learned what constitutes a good agent “harness.”

These agents are excelling in domains where the output is a “first draft” that a human can review and refine. This mitigates the risk of agents not being 100% reliable. Key applications include:

• Coding: This is the most mature domain, where agents create pull requests for human review.
• AI SRE (Site Reliability Engineering): Agents can investigate incidents by digging through logs and producing an initial report.
• Research and Report Generation: Agents can perform extensive research and compile initial drafts for financial analysis or content creation.
• Customer Support: Beyond simple first-line responses, agents can run in the background after a case is escalated to a human, compiling a comprehensive summary of the issue to assist the human agent.

The Architectural Shift: From Scaffolding to Harnesses

The architecture around models has undergone a significant evolution, moving from rigid structures to more flexible and powerful environments.

Three Eras of Agent Architecture

1. The Raw Model Era: Early models were simple text-in, text-out systems without tool-calling capabilities. Applications were limited to single prompts or simple, linear chains.
2. The Scaffolding Era: As models gained nascent tool-calling and planning abilities, developers built custom “cognitive architectures” or “scaffolding.” This involved more structured frameworks (like early versions of LangGraph) to guide the model through complex tasks, compensating for its reliability gaps.
3. The Harness Era: With the latest generation of models (e.g., GPT-4.5, Claude 3 Opus), a new architecture has emerged. The models are now capable enough to operate in a simple “LLM in a loop” fashion. The focus has shifted from rigid scaffolding to creating a rich, flexible harness.

What Defines a Modern Harness? Context Engineering

Everything about building a modern harness can be described as context engineering. It’s about cleverly managing what information the LLM sees at any given step. Key components of a harness include:

• Planning Tools: Opinionated, built-in tools that help the agent structure its approach.
• Context Compaction: As an agent runs for a long time, its context window fills up. Compaction is the strategy for summarizing or offloading information to preserve the most relevant context.
• File System Tools: Access to a file system (real or virtual) is critical. It serves as a scratchpad, a place to store large tool outputs without cluttering the context window, and a foundation for code execution.
• Sub-agents and Skills: Breaking down complex problems into smaller tasks handled by specialized sub-agents. A significant part of the harness’s job is prompting these components to communicate effectively with each other.

“A lot of a lot of hardness is context engineering… everything around contraction context engineering, sub agents context, skills context engineering. So we basically saw them using the same core algorithm but making just like improvements on context engineering and we’re like oh that’s interesting that’s pretty different than before.”

Building Agents: A New Software Development Paradigm

Building agents is fundamentally different from traditional software development because the application’s logic is not entirely contained within the code; a large, non-deterministic part of it resides within the model itself.

Traces as the New Source of Truth

In traditional software, the code is the source of truth. To understand its behavior, you read the code. With agents, this is impossible. The only way to understand what an agent is doing and why is to look at a trace of its execution.

• Debugging: Traces are indispensable for debugging. When an agent fails at step 14, the problem lies in the context built over the previous 13 steps. The debugging request is no longer “show me the code,” but “send me the LangSmith trace.”
• Testing: Traces become the raw material for creating test cases. Online testing with real-world inputs is more critical than offline unit testing, as emergent behaviors only appear in production.
• Collaboration: Traces are the central artifact around which teams collaborate to solve problems.

The Role of Evaluation and Human Judgment

Evaluating an agent’s output often requires human-level judgment, as they perform tasks previously done by humans.

• Human-in-the-Loop: Bringing humans in to annotate traces and provide feedback is a core part of the development loop.
• LLM-as-a-Judge: Using another LLM as a proxy for human judgment is a viable and scalable approach. The key is to calibrate the “judge” LLM against a dataset of human-labeled traces to ensure it aligns with human preferences (a concept LangSmith calls “Align Evals”).

The Future: Self-Improving Agents and Evolving Interfaces

The Central Role of Memory

Memory is essentially context engineering over longer time horizons. It allows an agent to learn from past interactions and improve its performance. This creates a powerful feedback loop that reduces the developer’s manual iteration burden.

A fascinating pattern emerging is recursive self-improvement. An agent can be given tools to pull down its own traces from a system like LangSmith, diagnose what went wrong in past runs, and then programmatically edit its own instruction files or prompts to correct the behavior. This is no longer theoretical; it’s a practical workflow being implemented today.

Are All Agents Coding Agents?

This remains a key open question.

• File Systems are Essential: Any long-horizon agent needs access to a file system for effective context management.
• Code is a General-Purpose Tool: Writing and executing code is an incredibly powerful and flexible way for an agent to accomplish a vast range of tasks.
• Specialization vs. Generality: While a general-purpose agent may ultimately be a coding agent, today’s coding agents are highly optimized for software development tasks and may not be immediately transferable to other domains.

The User Interface for Agents

Interacting with long-horizon agents will require new UI/UX paradigms:

• Async Management: For tasks that run for hours or days, users will need an asynchronous interface, similar to a Jira or Asana board, to manage multiple running agents.
• Synchronous Interaction: When an agent completes a task or needs feedback, the user will need to switch to a synchronous, chat-like interface to provide corrections or new instructions.
• Stateful Workspaces: Because agents modify state (e.g., files in a directory, a Notion page, a Google Drive), the UI must provide a clear view of this shared workspace where both the human and the agent collaborate.

• Tags:
• Software Development
• AI
• LLMs
• Context Engineering
• Feature
• Agents
• LangChain