Corrective RAG (CRAG) — Trust but Verify

Corrective RAG (Yan et al. 2024, arxiv 2401.15884) inserts a retrieval grader between retrieve and generate, then routes based on the grade.

The CRAG flow

1. Retrieve top-K from your vector store.
2. Grade each retrieved doc for relevance — correct, ambiguous, or incorrect. The grader is a small classifier (T5-base scale), prompt-tuned LLM, or cross-encoder.
3. Route:
   • All correct → use them, generate.
   • All incorrect → fall back to web search (Tavily, SerpAPI, Brave, You.com).
   • Mixed → combine internal correct + web results, then "knowledge refinement" (chunk further, drop noise) before generating.

Why this is high-leverage

It's the smallest change that fixes naive RAG's worst behavior: confidently generating from irrelevant retrieved chunks. Without a grader, the LLM trusts what's in context.

Implementation notes

• Grader can be a fine-tuned BGE-Reranker or a prompt-tuned Llama-3-8B. Cheap.
• Web fallback only fires on hard misses, so cost stays low.
• Knowledge refinement = strip-and-decompose: split each doc into 'strips', score per strip, keep only relevant strips. Reduces noise.

When to skip CRAG

• Closed corpora where web fallback isn't allowed (legal, medical, regulated).
• Domains where 'I don't know' is the right answer — just teach the generator to say so via prompt + eval, no grader needed.

How it relates to other variants

CRAG is a corrective layer. Self-RAG (Asai et al. 2023) goes further with reflection tokens that the model itself emits. Adaptive-RAG (Jeong et al. 2024) routes by query difficulty. They all share the principle: don't trust retrieval blindly.