GraphRAG: answering questions a single retrieved passage can't
Microsoft Research's GraphRAG pre-builds a knowledge graph and community summaries over a corpus, so a broad, corpus-wide question can be answered without depending on a single lucky retrieval hit.
AI-assisted / research-based
This field note was drafted with AI assistance and synthesizes publicly available research papers and disclosed industry practice on an emerging AI technique. It is not based on confidential deployment data, is not investment, legal, medical, or security advice, and every primary claim links directly to its source so you can verify it yourself.
Standard retrieval-augmented generation finds the passages most similar to a question and hands them to a model. That works well for a question with a specific, localized answer somewhere in the corpus. It works poorly for a question whose answer requires synthesizing information spread across the entire corpus — because no single retrieved passage contains it.
"What does this clause say" and "what are the recurring themes across everything we have" are different problems. Similarity search only solves the first one.
The distinction the paper names explicitly
From Local to Global: A Graph RAG Approach to Query-Focused Summarization ↗, by Edge and colleagues at Microsoft Research, states the gap directly: RAG fails on global questions directed at an entire text corpus — such as "what are the main themes in the dataset" — because this is inherently a query-focused summarization task, not an explicit retrieval task. Prior query-focused summarization methods, meanwhile, do not scale to the volume of text a typical RAG-indexed corpus contains.
Some questions need the whole corpus, not one passage
GraphRAG's answer builds a graph index in two stages, both computed once at indexing time rather than at query time: first, an LLM derives an entity knowledge graph from the source documents — the entities mentioned and the relationships between them; second, the system detects communities of closely related entities within that graph and pre-generates a summary for each community. At query time, each relevant community summary is used to generate a partial response to the question, and all partial responses are combined into one final answer.
Why pre-computation is the actual mechanism
The reason this scales where prior query-focused summarization did not is that the expensive step — reading and summarizing large swaths of the corpus — happens once during indexing, amortized across every future query, rather than being repeated for each new question. A query only has to combine pre-existing community summaries relevant to it, not re-read and re-summarize the underlying documents from scratch.
The paper reports this producing substantial improvements over a conventional RAG baseline specifically on global, sensemaking questions over datasets in the 1-million-token range — measured on both comprehensiveness and diversity of the generated answers. That scoping matters: the reported gains are for a specific class of broad question, not a general claim that graph-based retrieval outperforms standard RAG on every query type.
What this costs relative to standard retrieval
GraphRAG is a meaningfully heavier infrastructure investment than a standard vector index. Building the entity graph and community summaries requires an LLM pass over the corpus at index time, and — the part most likely to be underestimated — that index has to be refreshed as the corpus changes, or newly added entities and relationships simply will not be reflected in any community summary a query might draw on. A corpus that updates frequently needs a re-indexing strategy, not a one-time build.
The heavier index only pays off for global questions
What this changes in production
Enterprise knowledge assistants answering broad questions
"What are the recurring risk themes across this quarter's incident reports" or "summarize how our product positioning has evolved across every release note this year" are exactly the class of global, corpus-wide question standard chunk retrieval struggles with and GraphRAG's community-summary approach targets directly.
Research and due-diligence synthesis
Identifying patterns or relationships across a large set of source documents — rather than retrieving isolated facts from any one of them — benefits from the same local-to-global structure, provided the corpus is stable enough that the up-front indexing cost is worth paying.
Internal knowledge bases where relationships matter
Where the connections between entities are as important as the raw text describing any one of them — organizational structures, dependency chains, cross-referenced policies — the entity graph itself becomes a queryable asset, not just an intermediate step toward better retrieval.
A production checklist
Before adopting a graph-based RAG architecture, the team should be able to answer:
- Is the actual question type this system needs to answer global and corpus-wide, or would standard chunk retrieval already handle it — and has that comparison actually been tested?
- What is the re-indexing cadence, and does it match how frequently the source corpus actually changes?
- What is the additional indexing cost in LLM calls, storage, and latency-to-freshness compared with a standard vector index?
- Is community detection producing meaningful, coherent groupings on this specific corpus, or noisy ones that need tuning?
- Does the system fall back to standard retrieval for genuinely local questions, or force every query through the more expensive global path?
- Who owns validating that the entity graph itself is accurate, given it was extracted by an LLM rather than curated by hand?
Research referenced
- Edge et al. (2024), From Local to Global: A Graph RAG Approach to Query-Focused Summarization ↗.