Seven RAG Failures and How to Solve Them

Nate: Welcome, everyone, to today’s webinar on retrieval‑augmented generation (RAG). We’re recording this session, and you’ll get the link afterward. The video will also be on our YouTube channel within a few days. We’ll hold a Q&A at the end—submit questions in the chat or the Q&A widget and Michaela will answer as many as possible. Today’s topic is “Seven Ways RAG Fails and How to Fix Them,” presented by our ML Evangelist, Michaela Kaplan. Michaela, over to you. Michaela: Thank you, Nate. Hi, everyone—I’m Michaela Kaplan, ML Evangelist at HumanSignal. Let’s dive in. RAG can fail in two main stages: retrieval—working with the knowledge base, ranking, and context consolidation—and generation, where the language model produces the answer. A 2024 paper by Barnett et al. lists seven specific failure points. I’ll walk through each one and how to resolve it; you’ll receive a flowchart afterward. Retrieval issues first: (1) Missing Content—the answer isn’t in your knowledge base; fix by adding the documents and re‑indexing. (2) Missing Top‑Rank Documents—the correct document exists but isn’t in the top‑K; retrain the ranker with relevance labels and adjust K. (3) Consolidation Limits—the right doc is retrieved but dropped from the prompt window; adjust chunk size, summarization, or truncation, and experiment with K. Generation issues: (4) Not Extracted—the LLM ignores evidence and hallucinates; clean conflicting info, tighten prompts, and enforce citations. (5) Wrong Format—content is correct but structure (JSON, Markdown) is wrong; separate formatting directions from content or bind an output schema. (6) Incorrect Specificity—answer is too broad or too narrow; guide users with example questions, preprocess queries, or tweak prompts. (7) Incomplete Answers—multi‑part questions receive only partial responses; split compound queries into separate prompts and merge results. To diagnose quickly, use Ragas metrics: Context Precision (share of retrieved chunks that are relevant) and Context Recall (share of all relevant chunks that were retrieved). Low precision means missing or irrelevant docs; low recall means ranking or consolidation problems. Label Studio supports ranking tasks and error tagging; Graph RAG can enrich documents with metadata; query‑rewriting agents can refine user questions. Nate: Thank you, Michaela. Let’s open the floor to Q&A—please submit your questions now. Michaela: (Answers audience questions on chunking, trusting LLM explanations, metadata for Graph RAG, and related topics.) Nate: That wraps up today’s session. Thanks for joining, and thanks to Michaela for the insights. You’ll receive the recording and resources soon. Have a great day!