Definition
GraphRAG is retrieval-augmented generation that retrieves from a knowledge graph — entities, the relationships between them, and pre-written summaries of clusters of related entities — that an LLM has extracted from your documents, instead of (or alongside) fetching flat text chunks by vector similarity. The payoff is specific: it can answer global questions about an entire corpus, such as "what are the main themes across these documents?", that ordinary vector RAG structurally cannot, because the answer to a global question is not sitting in any single chunk waiting to be retrieved.
To see why that gap exists, look at what ordinary vector RAG actually does. It embeds each chunk of your corpus into a vector, and at query time fetches the handful of chunks whose embeddings sit nearest the question's embedding. That works beautifully when the answer lives in a few passages — "what dose of aspirin did the trial use?" — and fails quietly when it does not. Ask "what are the main themes across these 10,000 support tickets?" and top-k retrieval hands the model perhaps ten tickets; the other 9,990 never enter the context window, so the model summarizes a tiny sample and presents it as the whole. The question is really query-focused summarization, not retrieval, and no amount of better embeddings fixes a method that only ever looks at k passages. As the "From Local to Global" paper that introduced GraphRAG puts it, "RAG fails on global questions directed at an entire text corpus, such as 'What are the main themes in the dataset?'".
GraphRAG's move is to do the expensive reading once, at index time, and store the result as structure the model can reason over cheaply later. It reads the corpus with an LLM, records who and what appears and how they connect, groups the graph into themes, and writes a summary of each theme up front — so that a global question is answered by consulting a few hundred compact summaries rather than by trying to cram a million tokens of source text into one prompt.
How It Works
GraphRAG splits into a heavy indexing phase and a lighter query phase. The whole design is an asymmetry bet: pay a large one-time cost to build the index so that each later question is comparatively cheap.
Indexing uses an LLM "to build a graph index in two stages," in the paper's words:
- Chunk and extract. The documents are split into chunks (about 600 tokens each in the paper), and an LLM is prompted on every chunk to pull out entities (people, places, organizations, concepts), their attributes, and the relationships between them. Those become the nodes and edges of the graph. Because a single pass misses entities, GraphRAG runs repeated "self-reflection" passes over each chunk — GPT-4 extracted nearly twice as many entity references at a 600-token window as it did at 2,400 tokens, so smaller chunks plus extra passes deliberately trade cost for recall.
- Detect communities and summarize them. GraphRAG runs the Leiden algorithm on the entity graph — the paper uses Leiden "on account of its ability to recover hierarchical community structure" — to partition the graph into communities, clusters of densely connected entities, at several levels of a hierarchy (broad root-level communities at the top, fine-grained leaf communities at the bottom). An LLM then writes a summary, called a "community report," for every community at every level.
The output of indexing is a hierarchy of LLM-written summaries, each a compact précis of one theme in the corpus. Those summaries — not the raw text — are what most queries reason over.
Querying then runs in one of two main modes, distinguished cleanly in Microsoft's documentation by the shape of the question:
- Local search answers questions about a specific entity. It starts from that entity's node, pulls in its graph neighbors and the source chunks that mention it, and reasons over that focused bundle — "combining relevant data from the AI-extracted knowledge graph with text chunks of the raw documents." It is the right mode for "what are chamomile's healing properties?".
- Global search answers corpus-wide questions "by searching over all AI-generated community reports in a map-reduce fashion." In the map step, each community summary is sent to the LLM in parallel and produces a partial answer, and "the LLM is also asked to generate a score between 0-100 indicating how helpful the generated answer is"; partials scored 0 are filtered out. In the reduce step, the surviving partials are sorted by helpfulness score and packed into a final context window, which the LLM summarizes into a single answer.
A worked example: why the index is dear but each query is cheap
The reason to pay for indexing shows up at query time. Suppose you have a corpus of roughly 1.7 million tokens of news articles and you ask a global question. Answering it by brute-force map-reduce over the raw source text would feed the model on the order of the full 1.7 million tokens of context across the pass. Answering the same question over the root-level community summaries instead used only about 40,000 tokens — over 97% fewer. Across the paper's datasets, root-level summaries needed roughly 9x to 43x fewer context tokens per query than summarizing source text directly. The corpus was read once, into notes; the query only reads the notes.
Now put the other side of the ledger next to it. That same 1.7-million-token corpus, chunked at about 600 tokens, is roughly 2,800 chunks — and extraction is at least one LLM call per chunk, before any summarization, and more once the self-reflection passes run. Vector RAG would index the identical corpus with about 2,800 cheap embedding calls and zero LLM reasoning. That is the trade in one picture: GraphRAG spends thousands of LLM calls up front to make each global query affordable; vector RAG spends almost nothing up front and simply cannot answer the global query at all.
Real-World Applications
The reference implementation is Microsoft's open-source GraphRAG project, the system behind the 2024 "From Local to Global" paper from Microsoft Research. In that work it was evaluated on roughly 1-million-token collections of podcast transcripts and news articles, answering sensemaking questions such as "what are the main themes?" and "what insights can be gleaned about public health priorities based on news coverage?" — questions with no single ground-truth passage to retrieve.
Where GraphRAG earns its indexing cost is analyst-style sensemaking over a private corpus that no one has time to read end to end. A policy team asking "what themes recur across this quarter's incident reports?"; an investigator asking "how are these people and companies connected across the leaked documents?"; a support organization asking "what are customers complaining about across all our tickets?" — each is a global question whose answer is distributed across thousands of documents and cannot be pulled from any ten of them. These are exactly the queries where top-k vector retrieval returns a confident answer drawn from an unrepresentative sample.
Just as importantly, GraphRAG is deliberately not the tool for pinpoint lookup. "What is the SLA clause in contract 4471?" is answered faster and far more cheaply by vector RAG, because that answer really does live in one chunk. In practice, teams increasingly avoid the "graph versus vectors" framing entirely: hybrid retrieval (see hybrid-search) runs graph-based and vector-based retrievers side by side and routes each question to the one that fits, so a single system can serve both a themes-across-the-corpus question and a find-me-this-clause question.
Challenges
The honest headline is indexing cost, and it is structural rather than incidental. Because extraction is an LLM call per chunk and summarization is an LLM call per community per hierarchy level, the bill scales with the size of your corpus in LLM tokens, not in cheap embedding operations. Microsoft's own README leads with the warning, verbatim: "GraphRAG indexing can be an expensive operation, please read all of the documentation to understand the process and costs involved, and start small." That caution is the first thing the project tells you, and it is not marketing modesty — it is the defining property of the method.
Staleness follows from the same fact. Adding documents can shift the graph and invalidate community summaries, and the straightforward fix is to re-run extraction and re-summarize. Incremental indexing exists to avoid redoing everything, but keeping a graph current is a standing cost, not a one-time one — which matters most for exactly the live, growing corpora (tickets, news, filings) where global questions are most valuable.
Extraction quality also caps everything downstream. The graph is only as good as the LLM's entity and relationship extraction: merge two distinct people into one node, or miss a relationship, and every community summary built on top inherits the error, invisibly. The chunk-size and self-reflection knobs described above exist precisely because a single naive pass under-extracts, and tuning them trades cost against recall with no free setting.
Finally, even query time is not uniformly cheap. Global search is far lighter than reading source text, but it still touches many community summaries in its map step, so it costs meaningfully more than a single vector lookup. And there is a genuine design choice with no default answer: which level of the community hierarchy to answer from. Root-level communities give broad, cheap, low-detail answers; leaf-level communities give detailed, expensive ones. Picking the level is a breadth-versus-detail-versus-cost decision the operator has to make per use case.
Future Trends
The most active frontier is driving down the indexing bill that defines the method. Follow-up work from Microsoft, marketed as LazyGraphRAG, defers the costly LLM summarization step — doing far less work up front and more at query time — with the goal of capturing much of the quality at a fraction of the indexing cost. The direction matters more than any one system: anything that turns "read the whole corpus with an LLM before answering a single question" into a lazier, incremental process widens the range of corpora for which graph retrieval is affordable at all.
A second direction blends the two query modes rather than choosing between them. DRIFT search, in Microsoft's documentation, "introduces a new approach to local search queries by including community information in the search process," so a single query can zoom into a specific entity while staying aware of the broader themes around it — narrowing the gap between "local" and "global" that the original design drew sharply.
The broadest trend is the quiet merger of graph and vector retrieval. As hybrid systems mature, the interesting question stops being "graph or embeddings?" and becomes "which retriever should this question use?" — with graph structure handling connections and themes, vector similarity handling pinpoint recall, and a router deciding per query. GraphRAG's lasting contribution may be less the specific pipeline than the demonstration that some questions are summarization problems disguised as retrieval problems, and need an index built to match.