{
  "id": 21199504,
  "title": "2026-11-22: PagedAttention explained from scratch",
  "url": "https://urgent.news/2026/11/22/2026-11-22-pagedattention-explained-from-scratch",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-11-22T10:41:00.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/susheem-k/2026-11-22-pagedattention-explained-from-scratch-50fd"
  },
  "original_language": "en",
  "account": "This post breaks down Kwon et al.'s paper \"Efficient Memory Management for Large Language Model Serving with PagedAttention\" (arXiv:2309.06180). The study observes that in serving systems, only 20.4% to 38.2% of the KV cache memory is occupied by actual token states. Three causes for this waste are identified: allocating space for the maximum length, internal fragmentation within each allocation, and external fragmentation between allocations.\n\nPagedAttention, introduced by the paper and implemented in the vLLM server, draws inspiration from operating system paging. It functions by reserving only small, fixed-size blocks of memory for each request, maintaining a table to track where these blocks are located, and allowing requests that share text to point to the same blocks. These blocks are copied only when modifications are required.\n\nThe paper reports a 2-4x increase in throughput compared to FasterTransformer and Orca at the same latency. To illustrate this, the author walks through a single request using the OPT-13B model (40 layers, 5120 hidden size, fp16 numbers). A prompt of 100 tokens generating 50 tokens results in 150 tokens. The reserved memory for this request is 1.56 GB (1.677 TB), whereas the actual usage is only 117.2 MiB (92.7% of the reserved memory).\n\nTwo types of waste are discussed: unused slots and internal fragmentation. The unused slots represent wasted memory, while internal fragmentation occurs due to the over-provisioning of memory for the maximum length. External fragmentation arises from the presence of free memory in small, non-contiguous blocks. The paper emphasizes these two sources of waste as distinct, despite their interconnectedness.",
  "summary": "This post works through Kwon et al., \"Efficient Memory Management for Large Language Model Serving with PagedAttention\" (arXiv:2309.06180). The paper starts from a waste problem: in the serving systems it measured, only 20.4% to 38.2% of the KV cache memory held real token states. It names three causes: reservation of room for the maximum length, internal fragmentation inside each reservation,…",
  "key_points": [],
  "editors_take": "Adopting operating system paging principles transforms memory management for large language models by eliminating the massive waste caused by over-provisioning and fragmentation during the serving process.",
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}