Quando o RAG erra, quase nunca é o LLM: 4 falhas de recuperação e como logar cada uma
Todo problema de RAG parece problema do modelo. Quase nunca é. Demorei pra aceitar isso. Ficava trocando modelo de embedding, mexendo no prompt, achando que o Claude tava inventando coisa. O que estava quebrado era o que eu entregava pra ele, e só ficou óbvio quando comecei a logar o que voltava da recuperação. Depois que instrumentei, os erros se separaram em quatro tipos bem diferentes. 1.…
RAG systems often fail not because of the language model, but due to errors in the retrieval process. A reporter discovered this after implementing logging to track what was returned by the retrieval stage. Four distinct types of retrieval errors were identified.
1. Low scores indicating the question lacks an answer in the database, often overlooked. A simple fix is to use a threshold score, such as 0.7, to filter out non-existent information.
2. Neighboring chunks mistakenly substituted, like "reset password" and "reset product," which produce irrelevant results. Using hybrid search, combining vector and BM25 search, improves accuracy by weighting important tokens.
3. The model misinterprets the context and generates answers that extrapolate beyond the provided information. This requires clear system prompts to prohibit such extrapolation and request citations from the context.
4. Chunking errors, where text is split incorrectly, resulting in incomplete or malformed chunks. Proper chunking, respecting document structure, avoids these issues. Implementing detailed logging of chunk scores, input sizes, and retrieval results allows for faster identification and resolution of these problems.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
