Connecting the Dots with Context Graphs — Stephen Chin, Neo4j
Stephen Chin of Neo4j argues that traditional RAG is insufficient because AI agents lose the reasoning behind past decisions. He introduces Context Graphs as a solution to capture the 'why' behind decisions, creating a queryable system of precedent that provides grounded, explainable, and auditable results.
Stephen Chin from Neo4j presents the case for moving beyond simple data retrieval toward a “system of reasoning” for AI applications. The core problem with current AI systems is their lack of context. They operate on data siloed across disparate enterprise systems, Slack conversations, and customer threads. When asked to make critical business decisions, agents lack the interconnected knowledge to provide good answers, trapping developers in a “matrix” of disconnected information. The solution is to embrace a connected view using Knowledge Graphs and, more specifically, Context Graphs.
From Generic Answers to Grounded Insight
Knowledge Graphs provide a powerful way to aggregate information by modeling data as nodes (entities like people, companies) and relationships (connections like “lives with,” “drives”). By incorporating vector embeddings, these graphs can combine structured, factual data with the similarity search capabilities of LLMs.
This combination dramatically improves retrieval quality, a concept demonstrated with a healthcare example:
- Baseline LLM: When asked for a care plan for a patient’s emphysema, a standard LLM gives a generic, textbook answer.
- Vector RAG: A system using a vector database provides a slightly more specific, but still generic, recommendation like “respiratory therapy” and “deep breathing exercises.”
- Graph RAG: By grounding the query in a knowledge graph that connects the patient to their actual history, the system provides a highly specific and relevant plan. It recommends “smoking cessation counseling” and “pulmonary rehabilitation,” recognizing the patient has a history of smoking and has had a specific operation. The crucial context was lost in the similarity search of the vector database but preserved in the graph’s relationships.
Building AI Memory with Graphs
To achieve true reasoning, an AI needs a robust memory structure. Knowledge graphs are ideal for this because relationships are a first-class citizen, enabling highly performant traversal of complex data without costly table joins. This memory can be categorized into three types:
- Short-Term Memory: Captures the immediate context of an agent’s execution pipeline, such as the current conversation or state.
- Long-Term Memory: Organizes vast amounts of historical information, representing business processes, entities, and past user interactions to provide deep context.
- Reasoning Memory (Traces): This is the most critical layer. It stores the why behind past decisions. Instead of just getting a result from an LLM, this memory captures the thinking process, the policies applied, and the decision provenance, making the reasoning repeatable, auditable, and a source for learning.
The open-source Neo4j Agent Memory package is designed to implement these three memory types within a context graph structure.
Context Graphs in Action: A Financial Services Demo
Context Graphs are the practical application of these principles. They act as a dynamic, queryable audit log that captures the decision-making process, organizing it by entities and relationships and pulling knowledge from all connected sources.
An open-source financial services application demonstrates this power. When an agent is asked whether a loan for “Jessica Norris” should be approved, it doesn’t just perform a simple data lookup. Instead, it queries the context graph to build a complete picture:
- Retrieval: The system pulls her financial history, account information, and related margin trades.
- Precedent: It surfaces a previous loan rejection, a critical piece of information often lost in siloed systems.
- Risk Analysis: The graph traversal identifies connections to known fraud detection patterns.
- Explainability: The key feature is the visualization of the knowledge graph traversal. The human user making the final decision can see exactly what information the AI used—the prior rejection, the risk factors, the connected entities—to arrive at its recommendation to deny the loan.
This provides a grounded, auditable, and justifiable basis for the decision, allowing developers to build agentic applications that solve real business problems and that users can trust.