RAG Needs a Map: Using GraphRAG to Retrieve Connected Context — Nyah Macklin, Neo4j
A hands-on GraphRAG workshop from Neo4j showing how combining vector search with a knowledge graph gives an agent connected context, multi-hop traversal, and explainable answers. It covers the difference between semantic similarity and relationships, LLM-driven entity and relationship extraction under a fixed schema, building an Aura database and Python pipeline, and configuring an Aura agent with similarity search and text-to-Cypher, with a closing check that shows why visible query evidence matters.
Graph RAG Workshop: Connecting Retrieval to Knowledge Graphs
The workshop opens with a poll on who already uses RAG and graph-based RAG, then frames the session as a “Graph RAG 101” for newcomers. The speaker, a senior developer advocate at Neo4j with a background in graph theory, graph algorithms, and context engineering, positions the talk around making AI systems traceable and explainable, a concern that matters most in regulated domains like finance, medicine, and law.
Setup and Logistics
Attendees are asked to create a free GraphAcademy account, a free Neo4j Aura instance (explicitly not a paid one), and to copy the provided OpenAI API key from the workshop notes. The speaker stresses that all resources are free and that the courses are hands-on rather than lecture-based. Colleagues circulate to help with setup issues, and fallback credentials inside GraphAcademy are offered for anyone who cannot create an Aura instance in time.
Why Graphs Alongside Vectors
The conceptual core contrasts two representations of an apple: a vector of floats that is opaque to humans, and a knowledge graph in which the apple is a fruit, grows on a tree, is round and red, and is connected to other entities. The speaker argues that knowledge graphs make information simultaneously machine-readable and human-readable, and that a fusion of both is often the goal.
The critique of tabular data is pointed. Relational rows require complex joins to reveal how things relate, which slows applications down. A graph lets an agent hop node to node along relationships, tracing how a decision was reached. This becomes especially valuable when paired with agent memory, which the speaker mentions but defers to another talk.
The guidance on when to use which approach is practical. Vector RAG is usually sufficient for questions answerable by a search engine. Graph RAG becomes imperative for multi-hop questions (roughly three or more hops), aggregation, concatenation across documents, supply chain questions, or source code analysis. The speaker explicitly does not dismiss vector RAG and recommends using both, with graph augmenting rather than replacing the vector layer.
Architecture and Retrievers
A standard RAG pipeline chunks documents, embeds them, retrieves the top-k results, and passes them to an LLM. GraphRAG keeps this structure but includes a knowledge graph in the retrieval path. The speaker describes retrievers as the component that takes unstructured input such as a question and searches structured data for context. Neo4j supports full-text search, vector search, and Cypher-based retrievers. A news chatbot example illustrates a hybrid architecture: a news API for latest articles, a knowledge graph for historical relationships between topics, and a document store for textual reports.
The benefits named are richer context, improved accuracy, and explainability, particularly when traversal of the graph is combined with agent memory.
Knowledge Graph Construction
A knowledge graph is defined as an organized representation of real-world entities and their relationships, illustrated with colleagues as nodes connected by typed relationships such as “colleague” and “friend.” The construction pipeline gathers source text, chunks it, generates embeddings, extracts entities and relationships with an LLM guided by a prompt that requires JSON output, and then writes the graph. The speaker notes that manual extraction is possible but LLM-assisted extraction reduces time considerably. Cypher is introduced as the query language optimized for graphs, and a sample generated graph captures facts about Neo4j, the company, and its implementation in Java.
Hands-On Build
The hands-on portion moves into a GitHub Codespace with the open-source neo4j-graphrag-python package. Participants create a .env file from the provided example, paste in their Aura connection details and OpenAI key, and run a test script that verifies both the database connection and the API key. The structured knowledge graph builder then populates the Aura instance. The speaker demonstrates that re-running the builder causes duplication, which is why the database showed 30 lessons instead of 15 in a later check. Inspecting the graph with a MATCH query reveals nodes and relationships, and the Aura query interface doubles as an inspection tool, not just a visualization.
Building the Agent
Aura agents must be enabled at the organization level through settings, after which the speaker creates a “Graph RAG Fundamentals” agent tied to the instance, with a description and instruction grounding it in the GraphAcademy course. Vector search must be added before the agent can be saved. A vector index is created via Cypher, and a similarity search tool is configured against the chunk embedding index, returning the top five results with OpenAI’s text-embedding-3-small model at 1536 dimensions.
A retrieval Cypher query is attached so that results include connected graph nodes and similarity scores, which the speaker emphasizes is essential for using the graph. A natural-language-to-Cypher tool is also enabled, so the agent can translate questions into queries over the graph.
Testing and Verifying Evidence
The agent answers a question about temperature by describing it as a parameter controlling randomness in generated text and citing its sources. Jeremy points out a meta detail: the course PDFs the agent learned from were GraphAcademy’s own introductory GenAI materials. When asked how many lessons the course contains, the agent first reports 30, which is a duplication artifact from running the builder twice. Checking the underlying query in Aura returns the correct figure of 15. The speaker presents this as the reason visible evidence matters: even a working pipeline can produce plausible but wrong aggregate answers, and aggregation is a case where ordinary RAG tends to struggle.
Code Walkthrough
The builder code imports the Neo4j driver, the OpenAI LLM and embedder, a simple knowledge graph pipeline, and a fixed-size text splitter. Chunks are 500 characters with overlap to avoid losing context at boundaries. Schema is defined through example node types (technology, concept, example, process, challenge, benefit) and relationship types focused on “related to,” “used in,” and similar links, along with allowed patterns such as technology related to technology. The speaker stresses that this schema is what keeps the LLM from inventing new node and relationship types. For real business documents, the LLM can help draft a schema, but once fixed, it constrains extraction. The pipeline ingests CSV files and PDFs to construct the graph.
Closing Notes
The speaker summarizes the benefits as higher accuracy, easier development, and explainability, and points to GraphAcademy courses such as Cypher Fundamentals for further study. An in-person meetup was announced for July 1 in San Francisco, with the Neo4j team available at booth P3 to discuss use cases and the reasoning behind context graph decisions.