{
  "id": 4266817,
  "title": "How AI Stores Millions of Vectors Without Using Tons of Memory",
  "url": "https://urgent.news/2026/08/29/how-ai-stores-millions-of-vectors-without-using-tons-of-memory",
  "topic": "ai",
  "section": "AI",
  "published": "2026-08-29T19:55:59.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/rijultp/ever-wondered-how-ai-stores-millions-of-embeddings-47ek"
  },
  "original_language": "en",
  "account": "Rijul, the creator of LiveReview, a code review tool for critical business systems, explains how to store millions of vector embeddings efficiently without consuming large amounts of memory. This is crucial for applications like Retrieval-Augmented Generation (RAG) that rely on thousands of embeddings derived from product documentation, FAQs, and support articles.\n\nTo understand the problem, consider a customer support RAG app with 10 million embeddings, each consisting of 768 dimensions. Each dimension is represented by a 32-bit floating-point number, which is equivalent to 4 bytes. Thus, storing all 10 million embeddings would require approximately 30 GB of memory, equivalent to the storage capacity of a large game or several high-quality movies.\n\nProduct Quantization (PQ) is a technique that can significantly reduce the storage requirements for these large vectors. PQ works by splitting each vector into smaller pieces called subvectors and then learning representative patterns for each piece. These patterns are stored as \"codewords,\" allowing the entire subvector to be represented by the ID of its closest codeword.\n\nFor example, consider an 8-dimensional vector: [0.21, 0.73, -0.15, 0.91, 0.34, -0.52, 0.18, 0.66]. This vector can be split into four subvectors. For each subvector, a separate codebook is learned, and the subvector is replaced by the ID of the codeword that best matches it. The original vector is then represented by a list of these IDs, which can be stored using fewer bits than the original floating-point numbers.\n\nIn the example, the 8-dimensional vector is split into four subvectors, and each subvector is represented by an ID. The entire vector is now represented as [1, 2, 3, 0], where each number is the ID of the codeword for the corresponding subvector. This representation requires only 4 bytes, whereas the original vector required 32 bytes.\n\nWhen applied to larger embeddings, such as a 768-dimensional vector, PQ can reduce the storage requirement by a factor of 32. This dramatic reduction in storage significantly benefits applications that deal with millions or billions of vectors, making it feasible to perform large-scale vector search without consuming excessive memory.",
  "summary": "Hello, I'm Rijul, and I'm building LiveReview — a blast-radius aware AI code review built for your business-critical systems. Star us to help devs discover the project, give it a try, and share your feedback to help improve the product. In any sort of AI application, we have the concept of vectors . Vectors come in different shapes and sizes. Especially when they are large, we need a good way to…",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}