Why LLMs Hallucinate — And Why RAG exists

In a recent article, the author explains why large language models (LLMs) sometimes generate incorrect information, known as "hallucinations." The key point is that LLMs don't store and retrieve knowledge the way humans do, but instead predict the next token based on patterns learned during training and probabilities. This means they don't inherently verify truth, don't automatically know the data, and don't look up facts while generating responses.

As a result, when an LLM generates fake paper titles, believable author names, and academically correct citations, it appears convincing but is actually hallucinating. This issue becomes dangerous because the responses often sound fluent, well-structured, and highly confident, leading humans to associate confidence with correctness.

The author suggests that the real problem isn't intelligence, but the lack of context or grounding. Since LLMs don't have access to real-time information, company data, or documents by default, they can't verify the accuracy of their answers. The author proposes that providing relevant information beforehand, like asking a friend with access to the actual document, can help improve the quality of the response.

This leads to the concept of Retrieval-Augmented Generation (RAG), which is a system design shift that involves grounding generation using relevant external information before the model starts predicting. The next article will delve deeper into what RAG is, how retrieval changes generation, the limitations of simply adding a vector DB, and how modern AI systems retrieve useful context.