{
  "id": 14047426,
  "title": "GPUs on Kubernetes are bring-your-own-everything, starting with the bill",
  "url": "https://urgent.news/2026/10/12/gpus-on-kubernetes-are-bring-your-own-everything-starting-with-the",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-10-12T20:48:01.000Z",
  "source": {
    "name": "The New Stack",
    "slug": "the-new-stack",
    "url": "https://thenewstack.io/gpus-on-kubernetes/"
  },
  "original_language": "en",
  "account": "Running Kubernetes platforms often involves monitoring workloads, which can be done through Grafana dashboards. These dashboards show what containers requested versus what they used, allowing for easy identification of resource consumption by individual deployments. This method has been effective for CPU and memory resources for years. However, when it comes to GPUs, the visibility and management become more complex.\n\nGPU resource visibility requires creating custom metrics, as GPUs don't provide the same level of granular visibility as CPU and memory. NVIDIA's DCGM exporter, a popular tool for GPU metrics, reports at the physical device level by default. This approach works well when a single pod has exclusive access to a GPU, but it becomes problematic when multiple pods share a GPU through time-slicing or Multi-Instance GPU (MIG).\n\nWith time-slicing or MPS, DCGM provides the same value for all pods sharing a GPU, making it difficult to attribute resource usage accurately. This issue is further exacerbated by the fact that GPUs are billed as whole numbers, requiring precise matching of requests and limits without any overcommitting at the scheduler level. As a result, teams must build their own rightsizing math to determine how much of a GPU's capacity a workload actually used before addressing inefficiencies.\n\nAnother challenge with GPU resource management lies in determining GPU utilization. While CPU utilization is straightforward, GPU utilization is more nuanced. The traditional utilization metric indicates whether a kernel was running on the GPU during a sample window, but it doesn't provide information on actual compute or memory bandwidth consumption. GPUs can sit at 100% utilization while barely performing any meaningful work, and memory constraints often become the first signal of resource exhaustion.\n\nTo address these challenges, teams must develop their own definitions of utilization, focusing on metrics such as compute occupancy, memory footprint, bandwidth, or power consumption. By relying on these signals, organizations can make more informed decisions about resource allocation and billing. NVIDIA's NVML library can provide per-process memory usage on a GPU, but the data may not always add up to the total GPU capacity due to the driver reserving memory for its own use. In shared GPU environments, accurate attribution of resource usage becomes even more challenging, requiring careful consideration of billing strategies and the adoption of tools that can provide more accurate GPU utilization metrics.",
  "summary": "If you run a Kubernetes platform, you’ve probably got at least one Grafana dashboard for resource visibility on your workloads. The post GPUs on Kubernetes are bring-your-own-everything, starting with the bill appeared first on The New Stack .",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}