Mergeable by default: Building the context engine to save time and tokens — Peter Werry, Unblocked
A practitioner's guide to building a context engine, the reasoning layer that provides AI agents with the necessary organizational context to generate effective and appropriate code. The talk debunks common myths about RAG and large context windows, outlines core requirements for a robust context engine, and shares lessons learned from production.
The intelligence of AI models for code generation is rapidly increasing, but this intelligence is ineffective without the correct context. The real bottleneck for AI agents is not their ability to code, but their ability to access and reason over specific organizational context—the history, conventions, past decisions, and “battle scars” that an experienced developer possesses. Without this, agents produce code that costs an organization twice: once in tokens and again in long, painful review cycles.
A context engine is the reasoning layer designed to solve this problem. It’s responsible for supplying an agent with all the context it needs—and none that it doesn’t—in a highly optimized way, ensuring the agent’s work aligns with organizational best practices and expectations.
AI-generated code should just feel like it was written by someone that’s been in your team for like 20 years.
The Problem: Access Doesn’t Equal Understanding
As teams move from manually guiding agents to deploying parallel and background agents, the human becomes the bottleneck. A context engine is necessary to automate this guidance. However, simply giving an agent access to data sources is not enough. This leads to several common misconceptions.
Debunking Common Myths About Context
-
Myth: Naive RAG is a context engine. Standard retrieval-augmented generation runs into a critical issue known as satisfaction of search. Borrowed from radiology, this describes how an agent finds the first plausible-looking piece of information and stops searching, often missing the “golden nuggets” of context hidden in other sources like Slack conversations or incident reports. Naive RAG also struggles to resolve conflicts in data or personalize results for the user and their specific task.
-
Myth: Connecting many MCP servers is enough. Providing access to various tools and data sources does not grant the agent an understanding of the relationships between them. It cannot discern the “why” behind historical decisions or resolve conflicting information between, for example, outdated documentation and recent expert discussions.
-
Myth: A bigger context window will solve the problem. Even with million-token context windows, most organizations have far more data than can be loaded. More importantly, a large window doesn’t solve the fundamental challenge of reasoning across different, often conflicting, data sources within that context to determine what is true and relevant.
Core Requirements of a True Context Engine
A robust context engine must go beyond surface-level data access and provide deep, reasoned understanding. The goal is to surface the information “below the iceberg” of code that simply compiles.
- Unified System Context: It must build a knowledge graph that understands the relationships between disparate data sources like code, PRs, documents, and conversations. This includes distilling organizational memories and best practices from sources like historical pull request comments.
- Conflict Resolution: The engine must intelligently resolve conflicts between sources. This requires moving beyond naive strategies like recency and learning to weigh sources appropriately—for instance, prioritizing the main branch of code as a source of truth or valuing conversations between established experts.
- Targeted Retrieval and Personalization: Context must be relevant to the user, their team, and the task at hand. One effective technique is to perform a deep retrieval on repositories a user frequently contributes to while doing a wider, shallower retrieval on others, biasing results toward their primary work areas.
- Data Governance: It is critical to respect and enforce access controls. For example, when integrating with Slack, the engine can use information from private channels but must ensure that information is only surfaced to users who have permission to see it.
- Learning and Adaptation: The engine should learn from user feedback. When it surfaces information, it must provide references and allow users to provide feedback (e.g., “this is not helpful”), which helps the system refine its understanding over time, especially when validated by domain experts.
Hard-Won Lessons in Production
Building a context engine involves avoiding common pitfalls:
- Don’t Optimize for Access, Optimize for Understanding: Simply wiring up tools is insufficient. The real work is in building the deeper, relational understanding.
- Surface Conflicts Instead of Hiding Them: When the engine cannot resolve a conflict, it should surface the ambiguity to the user. This is a critical learning opportunity for both the user and the system.
- Never Cache Answers: Code, documentation, and organizational knowledge are constantly in flux. Caching answers quickly leads to serving stale and incorrect information. Using past answers as context for new ones can also cause the model to “regress towards a mean,” reinforcing its own potential errors.
High-Impact Applications
A context engine delivers the most value when integrated early and often in the development lifecycle.
- Planning Phase: This is where a context engine provides the “biggest bang for the buck,” ensuring a task is correctly scoped and understood from the start.
- Code Review: The engine can provide reviewers with the organizational context and motivation behind a change, not just the code itself.
- Other Use Cases: Ticket enrichment, incident triage and management (by connecting signals from tools like Datadog to code and past incidents), and powering engineering support channels to automatically answer recurring questions.