{
  "id": 17538598,
  "title": "GGUF VRAM calculator: the three numbers that decide",
  "url": "https://urgent.news/2026/11/01/gguf-vram-calculator-the-three-numbers-that-decide",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-11-01T05:22:22.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/mrsaynothing/gguf-vram-calculator-the-three-numbers-that-decide-2g8b"
  },
  "original_language": "en",
  "account": "The GGUF VRAM calculator reveals that three key numbers determine whether a quant fits within the available VRAM: weights, KV cache, and overhead. The formula calculates weights as parameters multiplied by 1e9, then by the quant bits-per-weight divided by 8, and then by 1.08. KV cache is calculated as twice the number of layers, KV heads, head dimensions, and a factor of 2 bytes. The overhead is the GPU's reserved resources, estimated between 0.6-1.2 GiB. The calculator, available as a downloadable file, uses real-world GGUF files from Hugging Face to test the accuracy, with an overall drift of just 0.04 GB. The KV cache is often overlooked but grows significantly with context size. For example, Llama-3 8B with 32 layers and 8 KV heads running at 32k context would exceed an 8GB card's capacity. The calculator highlights the importance of considering KV heads and context length, especially for older models, to avoid out-of-memory errors. It also points out blind spots like MoE sharding, draft models, and the impact of desktop usage on VRAM allocation. The tool is a simple, static page that emphasizes the importance of understanding these three numbers to predict VRAM usage accurately.",
  "summary": "Cross-posted from Mr Say Nothing — receipts live there. \"Will this quant fit?\" is a three-number question, not a feeling: weights + KV cache + overhead . I printed the formulas and shipped them as a calculator you can open from disk — then checked the math against four public GGUF files. Worst miss: 0.04 GB , under one percent. Here is the whole decision on paper, so the allocator never gets to…",
  "key_points": [
    "GGUF VRAM calculator identifies three critical numbers: weights, KV cache, and overhead",
    "Weights calculated by parameters, quant bits-per-weight, and 1.08 factor",
    "KV cache often underestimated but grows with context size, crucial for Llama-3 8B"
  ],
  "editors_take": "Accounting for KV cache and overhead alongside model weights provides a more accurate way to predict memory requirements and prevent out-of-memory errors during local large language model deployment.",
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}