GraphRAG Explained: When Knowledge Graphs Beat Vector Search

Use GraphRAG when your questions need relationships or a view across many documents, such as "which reports mention this supplier" or "what themes run through all 500 tickets". Use plain vector search when the answer sits in one passage. Most apps are mostly the second kind, which is why a graph is often the wrong first upgrade.

I wanted numbers rather than opinions, so on 2026-10-30 I built a tiny graph next to a vector index over a public-domain book and asked both the same questions. The corpus is small and my graph is hand-built, so treat the results as an illustration of the trade-off, not a benchmark.

What GraphRAG actually does

Microsoft's GraphRAG documentation (checked 2026-10-30) describes the pipeline this way: split the corpus into text units, extract entities, relationships and claims from each, cluster the graph into communities with the Leiden technique, and write summaries for each community. At query time it offers global search (answers holistic questions from community summaries), local search (starts at specific entities and explores their neighbours), DRIFT search (local search plus community context) and a basic vector search for questions where plain RAG is enough.

So there are two ideas folded together. One is a graph of entities linking chunks. The other is pre-computed summaries over groups of related entities. The first helps with "everything about X". The second helps with "what is this whole corpus about". If you only need the first, you can build a much smaller version yourself.

The docs page I read does not discuss indexing cost. Because extraction runs over every text unit, you should assume cost grows with corpus size and plan a small pilot first.

The test: one book, two retrievers

Setup, so you can judge it:

• Corpus: The Adventures of Sherlock Holmes from Project Gutenberg (public domain), 12 stories, cut into 876 chunks of about 120 words.
• Vector side: BAAI/bge-small-en-v1.5 embeddings via fastembed, cosine similarity, top-k.
• Graph side: a networkx graph linking each chunk to the entities it mentions. I did not use an LLM for extraction. I wrote alias lists for 16 characters and places and matched strings. A real pipeline would extract entities with a model and would be noisier, so this is the graph's best case.

Aggregate questions: "which stories mention X?"

For each entity I embedded "Which stories mention X?", took the top 10 chunks, and counted how many of the true stories those chunks (that actually mention X) covered.

Vector search recovered 18 of 27 story-entity pairs. The graph found all 27, but read that carefully: the truth was computed from the same alias lists, so the graph is right by construction. What the table shows is not that graphs are clever. It shows that top-k retrieval has a ceiling. Ten chunks cannot cover twelve stories, and for "Mrs. Hudson" only 1 of the 10 retrieved chunks even mentioned her. An entity index does not have that ceiling.

Single-fact questions: where the graph loses

Six questions, each with a distinctive answer phrase I located in the text beforehand (for example "five little dried orange pips", "cheetah", "cut your hair"). A hit means an answer chunk was in the retrieved set. Vector used top-5. The graph returned every chunk linked to the entity named in the question.

Vector hit 4 of 6, the graph neighbour set 3 of 6, and the graph returned a median of 14 chunks per question against 5 for vector. That is a lot of extra context to pay for and to distract the model.

Two things explain the graph's misses. The answer chunk often refers to a person by pronoun ("he", "the Doctor") so entity matching never links it. And "all chunks mentioning the entity" is not a ranking. A real system would rank the neighbours, which means you are back to vector similarity anyway.

When should you reach for a graph?

Decide from your questions, not from the hype:

1. Aggregate questions ("every contract that mentions a termination fee", "all incidents involving service X"). Often a metadata filter or a SQL table does this better than a graph. Try that first.
2. Multi-hop questions ("which customers use a product affected by this bug"). Here an explicit relationship store earns its keep, because no single chunk holds both facts.
3. Corpus-level summaries ("what are the main complaints across 2,000 reviews"). This is what community summaries are designed for. A map-reduce summarisation job is the cheaper alternative.
4. Single facts, policies, how-tos. Stay with vectors. Improve chunking, add hybrid search and reranking.

A cheap pilot: write 20 real questions, tag them single-fact, multi-hop or aggregate, and run your current retriever over all of them. If the failures cluster in the last two groups, a graph is worth a pilot. If they cluster in single-fact, it isn't.

Build the small version first

The experiment's core is about 20 lines. This is the shape (it ran on the Holmes corpus; the alias list is yours to write):

import networkx as nx

G = nx.Graph()
for ci, (story, text) in enumerate(chunks):
    for entity, aliases in ENT.items():
        if any(a in text for a in aliases):
            G.add_edge(("chunk", ci), ("ent", entity))

# aggregate question: which stories mention an entity?
stories = {chunks[n[1]][0] for n in G.neighbors(("ent", "Irene Adler"))}

For retrieval you would route a question to entities (an LLM call or a name matcher), pull the linked chunks, then rank them by embedding similarity and keep the top few. That hybrid is usually what "GraphRAG" ends up meaning in small projects. If your agent can choose between tools, expose "search chunks" and "look up entity" separately, as discussed in agentic RAG.

What this test cannot tell you

• The corpus is one 12-story book. Fiction is full of recurring named characters, which flatters entity graphs.
• Six single-fact questions is too few to claim vector search is better. Only the direction (the graph is not better here, and returns more text) is worth noting.
• No LLM extraction was run, so extraction errors, duplicates ("Mr. Wilson" versus "Jabez Wilson") and cost are not measured.
• I did not run Microsoft's GraphRAG package or its global search.

For measuring your own pipeline properly, see RAG evaluation metrics, and for fundamentals the RAG lesson.