Context Engineering 2.0: MCP, Agentic RAG & Memory // Simba Khadder

The central problem in building effective AI agents isn’t their inherent intelligence, but rather the challenge of providing them with the right data at the right time. Currently, we treat different sources of context—unstructured data via RAG, agent memory, and structured data via tools—as three separate, fragmented surfaces. This leads to complexity and “tool bloat.” The future, or Context Engineering 2.0, lies in unifying these disparate sources into a single, navigable context layer.

The Three Pillars of Context

Today’s agentic workflows rely on three main types of context, each with its own implementation and challenges:

1. Unstructured Data (RAG): Retrieval-Augmented Generation is effective for querying documents, PDFs, and knowledge bases. The process involves chunking text, creating embeddings, and performing a search. However, effective RAG is more complex than simple nearest-neighbor search; it often requires sophisticated hybrid queries to retrieve the most relevant context for the model.

2. Memory: Agent memory is crucial for maintaining state and user preferences. It can be broken down into:

   • Short-Term Memory: Retains the immediate context of a conversation, so the agent doesn’t have to re-process the entire history with every turn.
   • Long-Term Memory: Stores more permanent information, such as user preferences (“don’t use em-dashes”) or key concepts that should persist across multiple conversations or tools.

3. Structured Data: This is the most challenging and least-discussed area of context engineering. Current approaches are often flawed and unreliable.

Flawed Approaches to Structured Data

Two common methods for giving agents access to structured data are deeply problematic, especially in enterprise environments.

1. The Perils of Text-to-SQL

Allowing an agent to directly write and execute SQL against a database is a “mildly terrifying” idea for several reasons:

• Correctness: Agents can generate syntactically incorrect SQL.
• Ambiguity: Production databases are messy. An agent has no way to understand the internal knowledge and nuance required to distinguish between similarly named tables or columns.
• Security: Giving an agent direct query access to a production database creates enormous security and access control risks.
• Performance: A user could use prompt injection to coax the agent into writing a recursive or resource-intensive query that could take down the database.

2. The Failure of Wrapping APIs

Another common pattern is to wrap an existing REST API, turning every endpoint into an agent tool. This rarely works outside of contrived demos because:

• Tool Bloat: It creates an overwhelming number of tools for the agent to reason about.
• Parameter Complexity: API endpoints often have numerous parameters that provide flexibility for human developers but are too complex for an agent to navigate effectively in a single pass. This leads to wasted tokens and loops of trial-and-error.
• Low-Level Abstraction: REST APIs are not designed for agents. They are a low-level abstraction, often requiring multiple, chained calls to accomplish a single, meaningful task.

The Solution: A Unified, Schema-Driven Context Engine

Instead of giving agents low-level access to databases or APIs, we should provide a higher-level, schema-driven abstraction. Inspired by ORM (Object-Relational Mapping) frameworks like Pydantic, this approach involves defining your data at the business-logic layer.

You define your core entities (Customer, Order, Item), their attributes, and the relationships between them. This schema is then exposed to the agent as a single, navigable data model.

In practice, an agent’s first step is to explore this model to understand what data is available. It can then navigate the relationships directly—for instance, listing a customer’s orders and then traversing to the items within each order. This is a far more intuitive and efficient process than chaining API calls or guessing at SQL table structures.

The Vision: A Single Pane of Glass for Context

Context Engineering 2.0 treats all data sources—APIs, data warehouses, user inputs—as a unified surface. The goal is to create a “materialized view of context” where structured data, unstructured text, and memory are all part of the same navigable space. An agent should be able to blend semantic search over documents with structured filters on data in a single, coherent call.

This unified layer provides significant benefits:

• Observability: When all context retrieval flows through a single point, it becomes dramatically easier to observe what an agent is doing and why.
• Security: By creating a materialized view, you explicitly control what data is exposed to the agent, creating a clean, secure boundary and preventing clever agents from finding ways around internal access controls.

Ultimately, context engineering and feature engineering are two sides of the same coin: creating useful signals for models. While they may exist as separate platforms in practice today—with feature stores serving classical ML and context engines serving agents—the underlying principle is the same. The future of AI is building a unified, semantic layer that connects agents to all of an organization’s data in a reliable and scalable way.

• Tags:
• AI Agents
• LLM
• RAG
• Context Engineering
• Feature
• Memory
• Structured Data
• Redis