Naive RAG — The Baseline

Naive RAG is five steps. Memorize them.

1. Ingest. Load source documents. PDFs, HTML, Markdown, transcripts. The format determines your chunking.
2. Chunk. Split into ~256–800 token chunks. Use RecursiveCharacterTextSplitter for prose, sentence-window for FAQs, layout-aware (e.g., unstructured, Docling) for PDFs with tables.
3. Embed. Run each chunk through an embedding model. Store the vectors plus a copy of the chunk text and metadata (source URL, page, section).
4. Retrieve. At query time, embed the user query, find top-K nearest chunks by cosine similarity (or dot product if vectors are normalized).
5. Generate. Concatenate retrieved chunks into the prompt as context, plus a strict system instruction ("answer only from context"), then call the LLM.

Where naive RAG fails — and why every advanced technique exists:

• Query–corpus vocabulary mismatch. User says "cancel my plan", docs say "subscription termination policy". → fixed by query rewriting / HyDE.
• Chunks lose context. A definition lives on page 3, a usage example on page 47. → fixed by parent-document retrieval, larger windows, hierarchical indexing.
• Top-K is noisy. Number 7 in your top-10 is irrelevant and confuses the model. → fixed by re-ranking.
• Long context fails. The model ignores the middle of a long context window ("lost in the middle", Liu et al. 2023). → fixed by context compression, smarter top-K.
• No metadata filtering. User asks about 2024 docs, you retrieve 2019 ones. → fixed by metadata filters at retrieval time.

When naive RAG is enough: small corpus (<10K chunks), single source type, single language, prose-heavy content, low-stakes (chatbot over docs site). Don't over-engineer until eval shows you need to.

Cost shape. Index time: O(corpus × embed-cost). Query time: O(K × generate-cost) where K is your top-K plus query embed (one cheap call).