{
  "id": 3347562,
  "title": "The sm_120 shared-memory cliff: why FP8 KV cache crashes vLLM on workstation Blackwell",
  "url": "https://urgent.news/2026/08/25/the-sm-120-shared-memory-cliff-why-fp8-kv-cache-crashes-vllm-on",
  "topic": "ai",
  "section": "AI",
  "published": "2026-08-25T21:02:33.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/conatusai/the-sm120-shared-memory-cliff-why-fp8-kv-cache-crashes-vllm-on-workstation-blackwell-1dji"
  },
  "original_language": "en",
  "account": "The shared-memory issue affecting vLLM on workstation Blackwell cards stems from the fp8 KV cache crashing when the GPU reaches its shared memory limit. DeepSeek-family models such as MLA models running on RTX PRO 6000 or any workstation or consumer Blackwell card may experience this issue on startup, resulting in \"triton.runtime.errors.OutOfResources: out of resource: shared memory\" error. Reducing block sizes or the number of stages may help alleviate the problem. The hardware fact behind this issue is that datacenter Blackwell parts advertise a larger per-block shared-memory budget compared to workstation and consumer silicon. On an RTX PRO 6000 Blackwell (sm_120) GPU, the shared memory per block is capped at 101,376 bytes. When configuring the kernel for vLLM, launching with a specific tile configuration and sweeping the `num_stages` parameter can help determine the cause of the crash. It was found that with bf16 KV, adding an extra pipeline stage costs about 4 KB, while with fp8 KV, a single stage costs about 18 KB, and stage two consumes the full per-SM budget, exceeding the per-block capacity by 1,024 bytes. The recommended solution to this shared-memory cliff is to compute the tile's actual shared-memory requirement and compare it to the shared_memory_per_block_optin for the specific device being used. This approach handles various Blackwell cards and similar hardware without requiring a special case for each chip.",
  "summary": "If you run vLLM with --kv-cache-dtype fp8 on a DeepSeek-family (MLA) model and your GPU is a GB10, an RTX PRO 6000, or any workstation or consumer Blackwell card, there is a decent chance the engine dies on startup with: triton.runtime.errors.OutOfResources: out of resource: shared memory, Required: 102400, Hardware limit: 101376. Reducing block sizes or `num_stages` may help. We hit this,…",
  "key_points": [
    "vLLM crashes on Blackwell cards due to fp8 KV cache exceeding shared memory limit",
    "Recommended solution: compute tile's shared-memory requirement vs device's sharedmemoryperblockoptin"
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}