Beyond Prompting: The Emerging Discipline of Context Engineering Reading Group

The field of Large Language Models (LLMs) is shifting from the relatively simple concept of prompt engineering to a more comprehensive and systematic discipline known as Context Engineering. While an LLM’s capabilities are powerful, they are fundamentally governed by the context provided. As we demand more complex reasoning and actions from these models, they in turn demand a more nuanced and structured approach to how we feed them information. This survey paper organizes the vast research landscape into a coherent framework, distinguishing the foundational building blocks from their practical implementations.

From Prompt Engineering to Context Engineering

The core distinction lies in how we conceptualize “context.”

• Prompt Engineering: Historically, context (C) was treated as a monolithic, static string of text fed to a model.
• Context Engineering: This paradigm reconceptualizes context as a dynamically structured and assembled set of diverse components. These components can include user instructions, external knowledge, available tools (function calling), persistent memory, and the state of the system or user. An “assembly function” orchestrates these pieces into the final context fed to the model.

This entire process is an optimization problem, constrained by the primary challenge of the Transformer architecture: the quadratic computational and memory overhead imposed by the self-attention mechanism as sequence length increases.

The Foundational Components of Context Engineering

The paper presents an analogy to understand the core building blocks of context engineering: sourcing ingredients, cooking them, and managing the pantry.

1. Context Generation & Retrieval (Sourcing the Ingredients)

This component is about gathering the raw materials for the context.

• Advanced Prompting: Research has evolved beyond simple zero-shot or few-shot prompting to more complex cognitive structures that guide the LLM’s reasoning process.
  • Chain of Thought (CoT): Decomposes a problem into intermediate reasoning steps.
  • Tree of Thoughts (ToT): Explores multiple reasoning paths, allowing the model to backtrack from dead ends.
  • Graph of Thoughts (GoT): A more sophisticated topology where the model can form a complex network of thoughts, merging ideas from different reasoning chains, much like human ideation.
• External Knowledge Retrieval (RAG): This addresses the need to incorporate non-parametric knowledge (information not embedded in the model’s weights). Research in Retrieval-Augmented Generation is active and includes methods like:
  • Decompositional RAG: Breaking a complex query into sub-questions and performing retrieval for each.
  • Recursive RAG: Building a tree of text chunks with varying levels of summarization to provide context at multiple granularities.
  • Graph-Enhanced RAG: Retrieving facts from knowledge graphs to prepend to the prompt, providing structured, relational information.

2. Context Processing (Cooking the Ingredients)

Once the ingredients are sourced, they must be processed effectively, primarily to manage the constraints of the context window.

• Architectural Innovations: A major line of research questions the Transformer architecture itself, exploring sub-quadratic time alternatives like Mamba (State Space Models) and other linear attention or recurrent models to handle long sequences more efficiently.
• Iterative Refinement: Techniques like Self-Refine involve a feedback loop where an LLM’s initial output is passed back to itself for critique and improvement, effectively “cooking” the information more thoroughly.
• Multimodal Integration: Research is focused on enabling LLMs to process and relate information across different modalities (text, image, etc.). This includes cross-modal attention mechanisms that learn dependencies between textual and visual tokens directly within the model’s embedding space.

3. Context Management (Managing the Pantry)

This component deals with memory and deciding what information to keep in the limited context window, what to offload, and what to discard.

• Virtual Context Management: Inspired by operating systems, techniques like MemGPT create the illusion of a much larger memory by paging context between the active (physical) context window and an external (disk) memory store.
• Architectural Augmentation: Methods like Camelot augment existing LLM architectures by adding an Associative Memory (AM) module to each layer. This allows the model to efficiently search for and retrieve relevant memorized tokens from a memory bank during generation.

Advanced Implementations

Building on these foundations, several advanced systems have emerged.

RAG and Memory Systems

Retrieval-Augmented Generation (RAG) has evolved from naive “spray and pray” vector searches to highly structured systems.

• Modular & Agentic RAG: These systems use LLM-based agents to intelligently decompose queries, determine which data sources to query, and decide whether retrieval is even necessary.
• Graph-Based RAG: This approach, exemplified by tools like LightRAG, structures knowledge into a graph database of entities and relationships. This provides the LLM with highly structured information it can traverse and reason over more effectively. The process involves using an LLM to extract entities and relationships from documents, building a knowledge graph, and then using a combination of vector search and graph traversal for retrieval.

Memory Systems add a crucial element that RAG lacks: statefulness.

RAG is the agent’s I/O into the world. Memory is the agent’s internal state.

While RAG is stateless, memory allows an agent to retain knowledge about past interactions. Tools like MemZero manage this by building a memory graph of facts. When new information arrives, the system checks for conflicts, updates the graph, and uses this stateful knowledge to provide personalized, context-aware responses. This is critical for applications like personal assistants that need to remember user preferences over time.

Tool-Integrated Reasoning & Multi-Agent Systems

• Tool-Integrated Reasoning: This is the capacity of an LLM to interact with the external world by using tools, transforming it from a “text generator to a world interactor.” This goes beyond simple function calling to include reasoning about which tools to use, when to use them, and how to chain them together to accomplish a complex task. For example, a site reliability engineering (SRE) agent might use a sequence of tools: read an alert from Slack, fetch logs, analyze source code from GitHub, and then create a pull request.
• Multi-Agent Systems: This concept involves creating an architecture of multiple specialized agents that communicate and collaborate to solve problems. This can be seen as “Conway’s Law on purpose,” where the system’s architecture deliberately mirrors the division of labor required for a task. Each agent has its own specialized knowledge, context, and tools. Communication protocols like MCP (Meta-Call Protocol) and the concept of “Agent Cards” (which describe an agent’s capabilities) are being developed to facilitate this coordination. The challenges in orchestrating these multi-agent systems often mirror the well-known challenges in distributed software systems.

• Tags:
• Large Language Models
• RAG
• Context Engineering
• Feature
• Multi-Agent Systems
• Tool Use
• Memory Systems