{
  "id": 19144628,
  "title": "Podcast: Distributed LLM Inference on Kubernetes with llm-d, Wasm, & TornadoVM",
  "url": "https://urgent.news/2026/11/09/podcast-distributed-llm-inference-on-kubernetes-with-llm-d-wasm",
  "topic": "ai",
  "section": "AI",
  "published": "2026-11-09T11:00:00.000Z",
  "source": {
    "name": "InfoQ",
    "slug": "infoq",
    "url": "https://www.infoq.com/podcasts/distributed-llm-inference-kubernetes/"
  },
  "original_language": "en",
  "account": "In this episode of InfoQ Podcasts, Edoardo Vacchi, a Red Hat engineer, discusses the evolving landscape of WebAssembly in backend environments, distributed AI inference using llm-d and vLLM on Kubernetes, and running Java workloads on GPUs via TornadoVM. Edoardo explains how llm-d manages cache locality, request routing, and batch inference across Kubernetes clusters to optimize GPU utilization. He also touches on the WebAssembly ecosystem's evolution, Wasm's shift towards a modular system, and the growing importance of sandboxing in today's development landscape.",
  "summary": "In this episode, Edoardo Vacchi discusses the evolving landscape of WebAssembly on the backend, running Java workloads on GPUs using TornadoVM, and optimizing distributed AI inference using llm-d and vLLM on Kubernetes. Edoardo explains how llm-d manages cache locality (KV and prefix caches), request routing, and batch inference across Kubernetes clusters to maximize expensive GPU utilization. By…",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}