{
  "id": 15756382,
  "title": "A Dense 31B on an 8 GB GPU: Gemma 4 31B on an RTX 3050",
  "url": "https://urgent.news/2026/10/20/a-dense-31b-on-an-8-gb-gpu-gemma-4-31b-on-an-rtx-3050",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-10-20T15:18:06.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/gde/a-dense-31b-on-an-8-gb-gpu-gemma-4-31b-on-an-rtx-3050-26p9"
  },
  "original_language": "en",
  "account": "This article provides a detailed guide on how to serve Gemma 4 31B, Google's quantization-aware trained (QAT) dense model, on an 8 GB NVIDIA RTX 3050 GPU using llama.cpp. The model file is 17.7 GB and cannot fit entirely on the GPU, but llama.cpp manages to run as many layers as possible on the GPU and the rest on the CPU. The article also discusses the performance of the 31B model compared to its 12B and 26B siblings, with the 31B being the most accurate but also the slowest. The measures include generation speed, context length, drafter additions, and whether the answers surpass previous models. The article is the sixth part of a series, with the previous parts covering Gemma 4's MTP Drafter, E2B and E4B measurements, the context size for a 12B on an 8 GB GPU, and the 26B mixture-of-experts running on an 8 GB GPU.",
  "summary": "This article provides a step by step guide to serving Gemma 4 31B, Google's quantization-aware trained (QAT) dense model, on an 8 GB NVIDIA RTX 3050 with llama.cpp. The model file is 17.7 GB, so it cannot fit on the card. llama.cpp puts as many layers as will fit on the GPU and runs the rest on the CPU. The measures are how fast it generates, how much context fits, what the drafter adds, and…",
  "key_points": [
    "Gemma 4 31B model file is 17.7 GB, exceeding 8 GB GPU capacity",
    "llama.cpp runs as many layers as possible on GPU, rest on CPU",
    "31B model is most accurate but slowest compared to 12B and 26B"
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}