Graph RAG indexes a corpus as a knowledge graph — nodes (entities) and edges (relationships) — instead of (or alongside) vectors.
The core insight: vector RAG is good at "what does this say", graph RAG is good at "how is this connected".
Microsoft GraphRAG architecture (2024)
- Entity extraction. LLM extracts entities, relationships, and claims from each chunk.
- Graph construction. Build a typed graph: entity nodes with descriptions, edges with relationship descriptions.
- Community detection. Run Leiden algorithm to find clusters of densely connected entities.
- Hierarchical summarization. LLM summarizes each community, then summarizes summaries up the hierarchy.
- Query. Two strategies:
- Local search: find relevant entities, walk neighbors, retrieve attached chunks.
- Global search: over community summaries — answers questions about the dataset as a whole.
When GraphRAG wins
- Multi-hop reasoning. "Who at IBM has worked on both watsonx and Arabic NLP since 2022?" — a vector retriever with no entity model can't easily intersect these.
- "Whole-dataset" questions. "What are the dominant themes in 5000 customer interviews?" — vector RAG can't aggregate; community summaries can.
- Connected/structured data. Org charts, supply chains, citation networks, code call graphs.
When GraphRAG loses
- Cost. Indexing a 10MB corpus runs ~$50–200 in LLM calls for entity extraction and summarization.
- Slow updates. Adding a doc means re-extracting, possibly re-clustering. Not real-time.
- Free-text Q&A. For "what does this say about X?", vector RAG ties or beats it at a fraction of the cost.
Practical take
In production, hybrid graph + vector wins more than pure graph. Use the graph for entity-grounded queries, vector for free-text. Microsoft's reference implementation supports both modes.