Phase 2: Advanced RAG
Fix RAG answers that are only okay. The basic pipeline works on easy questions and fails on hard ones. This phase teaches which lever to pull: reorder results, rewrite the question, use relationships between documents, keep the index fresh, and measure whether any change actually helped.
After this phase you can
- Rerank retrieved chunks and see the order improve
- Rewrite or expand a query before searching (including HyDE)
- Measure RAG quality with hit rate, MRR and an LLM judge
Before you start: Phase 1: you can chunk, embed and retrieve the top-k chunks.
Lessons in this phase
- Reranking Intermediate · 8 min
Reranking is a second, more careful pass that re-orders the chunks your first search returned, so the best ones end up at the top.
- Cross Encoder Intermediate · 8 min
A cross-encoder is a model that reads the question and a document together and outputs one relevance score for that pair.
- Bi Encoder Intermediate · 8 min
A bi-encoder turns the question and each document into vectors separately, then finds matches by comparing those vectors.
- Context Compression Intermediate · 8 min
Context compression trims retrieved chunks down to only the parts that help answer the question, before they go into the prompt.
- Query Expansion Intermediate · 7 min
Query expansion adds related words or phrasings to the user's question so the search finds documents that use different words for the same thing.
- Multi Query Retrieval Intermediate · 8 min
Multi-query retrieval asks an LLM to rewrite one question into several different search queries, runs them all, and merges the results.
- HyDE Advanced · 8 min
HyDE (Hypothetical Document Embeddings) asks an LLM to write a fake answer to the question, then searches with that fake answer instead of the question.
- Step Back Prompting Advanced · 6 min
Step-back prompting first asks a more general question about the underlying concept, retrieves for that too, and uses both to answer the specific question.
- Self Query Retriever Intermediate · 7 min
A self-query retriever uses an LLM to split a natural-language question into a search phrase plus structured metadata filters, then runs both.
- Multi Vector Retriever Advanced · 7 min
A multi-vector retriever stores several different vectors for each document (summary, questions it answers, small pieces) but returns the full original document when any of them matches.
- Parent Document Retriever Intermediate · 6 min
A parent-document retriever searches over small chunks for accuracy, then returns the larger "parent" section each chunk came from, so the LLM gets full context.
- Ensemble Retrieval Intermediate · 7 min
Ensemble retrieval runs several different retrievers (usually keyword and vector) on the same question and fuses their results into one ranked list.
- Knowledge Graph RAG Advanced · 8 min
Knowledge-graph RAG stores facts as connected entities and relations, and retrieves by following those connections instead of (or as well as) matching similar text.
- GraphRAG Advanced · 9 min
GraphRAG builds an entity graph from your documents, groups it into communities, pre-writes a summary for each community, and uses those summaries to answer broad "what are the main themes?" questions.
- Multi Modal RAG Advanced · 8 min
Multi-modal RAG retrieves not just text but also images, tables, charts, audio or video, and gives them to a model that can understand them.
- SQL RAG Advanced · 9 min
SQL RAG (also called text-to-SQL) lets an LLM turn a question into a SQL query, runs it on your database, and uses the result rows as context for the answer.
- API RAG Advanced · 8 min
API RAG fetches live data from an API at question time (weather, order status, stock levels) and gives the response to the LLM as context.
- Incremental Indexing Intermediate · 8 min
Incremental indexing updates your RAG index by re-embedding only the chunks that are new or changed, and deleting chunks that disappeared, instead of rebuilding everything.
- Streaming Retrieval Advanced · 8 min
Streaming retrieval means sending results and answer tokens to the user as soon as each piece is ready, and keeping the index updated continuously as new data arrives, instead of waiting for everything to finish.
- RAG Evaluation (RAGAS / DeepEval) Advanced · 11 min
RAG evaluation measures, with numbers, whether your system retrieves the right context and writes answers that are correct, grounded in that context, and on-topic.
- LLM-as-a-Judge Advanced · 10 min
LLM-as-a-judge uses a language model to grade answers when there is no exact match to check, and it only works if you design a rubric, fight the judge's known biases, and measure how often it agrees with people.