Context Engineering for Coding Agents

This talk presents a framework for “Complex Engineering for Coding Agents,” focusing on managing the context window of Large Language Models (LLMs) like Claude to solve complex, real-world problems. The speaker, Fauso, argues that while the context window is growing, effective engineering requires treating it as a scarce resource. He suggests a rule of thumb: keep context usage below 25% of the model’s capacity to maintain speed, reduce cost, and minimize errors.

A Taxonomy of Context Injection

The core of the presentation is a detailed breakdown of how to influence an agent’s behavior through various forms of context injection, categorized as deterministic, probabilistic, and human-driven.

• Deterministic Context: These are predictable injections that load at the start of a session.

  • claude.md files: Hierarchical markdown files for user-level, project-level, or folder-level instructions.
  • Rules: Path-scoped or file-extension-scoped instructions for coding conventions and standards.
  • Hooks: Scripts tied to lifecycle events, useful for tasks like summarizing a session upon completion or using an observer model to refine user input in real-time.

• Probabilistic Context: These are more dynamic and are invoked based on the flow of the conversation.

  • Sub-agents: Specialized agents for non-coding tasks (e.g., research, business operations) that do not inherit the parent agent’s full context, allowing for more focused and efficient execution.
  • Skills: Reusable slash commands that can range from simple markdown instructions to complex scripts that call other models or APIs (e.g., using Gemini for video analysis). The speaker emphasizes creating skills dynamically from successful chat interactions.
  • Observers: Separate processes that monitor a session and can interact with it. An example given is using tmux to allow different agent sessions to communicate with each other.

The Brain as a Mental Model for Agent Architecture

To manage the complexity of these systems, the speaker proposes using the human brain as a mental model, focusing on attention and memory.

• Attention & Working Memory: The LLM’s context window is analogous to human working memory—a limited space where active processing occurs.
• Long-Term Memory: To overcome the limitations of the context window, a long-term memory system is proposed. Inspired by Andrej Karpathy, this system uses a “wiki” of Markdown files as a knowledge base. Key components include:
  • Storage: An agent, or a team of agents, processes information (e.g., research papers, documentation) and stores it in the wiki.
  • Retrieval: An observer agent monitors the main chat and fetches relevant information from the wiki to inject into the context window at the right moment.
  • Maintenance: A policy-based system maintains the wiki’s relevance. This includes a scoring mechanism for concepts and a decay rule (like Ebbinghaus’s forgetting curve) to downgrade the importance of unused information over time.

This system allows the agent to build a persistent, transferable knowledge base, enabling it to solve problems more effectively, especially under time constraints, as demonstrated with a cellular automata visualization task.

The Build-off Challenge

The talk culminates in a hands-on challenge where teams must extract structured information from a technical drawing within a tight, five-minute timeframe. The challenge is designed to test the application of the discussed concepts, forcing participants to build a robust system with a memory wiki and specialized skills to solve a problem that is intractable with simple prompting or research alone.

• Tags:
• Prompt Engineering
• LLM
• Feature
• Claude
• Context Management
• Agentic Systems
• Knowledge Management
• Coding Agents
• Memory Systems