LLM - Day 5 - RAG a Intro

Retrieval Augmented Generation (RAG) is a technique that utilizes three main components to process private data sources, such as company Confluence data or documents, which are not publicly available. The first component is pre-processing steps, which transform the raw data into a structured format. The second component is the embedding model, which converts the pre-processed data into a multi-dimensional space, represented by vectors or embeddings.

The third component is the vector database, which stores these vectors for efficient querying and retrieval.

The pre-processing step involves breaking down the documents into smaller chunks, typically around 500 characters each. These chunks serve as individual units for further processing. The embedding model, which is essentially a large language model (LLM) containing a Transformer architecture, generates a vector or point for each chunk. These points are then visualized in a graph, with similar topics clustering together based on their proximity in the multidimensional space.

To retrieve relevant information, a query is provided to the vector database, along with the stored vectors. The query, represented by an orange X point, searches for the top-k most similar elements, using distance metrics such as Euclidean or Manhattan distance, or cosine similarity. The cosine similarity metric is particularly useful, as it considers the semantic relationship between the query and the stored vectors. In simple terms, a smaller angle between two points indicates a higher similarity or closeness.

The retrieved data, along with the context from the vector database, is then combined to form an augmented prompt. This prompt is fed into a language model, such as ChatGPT or Gemini, to generate a coherent response based on the provided information. This process allows RAG to effectively handle dynamic documents and provide accurate, relevant answers to queries, even when dealing with private data sources.