{
  "id": 20613406,
  "title": "AI’s hidden energy tax: More than 80% of scale-up traffic is just memory moving around",
  "url": "https://urgent.news/2026/11/18/ais-hidden-energy-tax-more-than-80-of-scale-up-traffic-is-just-memory",
  "topic": "ai",
  "section": "AI",
  "published": "2026-11-18T14:57:55.000Z",
  "source": {
    "name": "SiliconANGLE",
    "slug": "siliconangle",
    "url": "https://siliconangle.com/2026/11/18/memory-bottlenecks-ai-how-photonics-shared-pools-help-sc26/"
  },
  "original_language": "en",
  "account": "AI infrastructure is facing costly memory bottlenecks as inference takes over from training. GPUs are idle while high-bandwidth memory, or HBM, remains stuck to processors that cannot share it. Marvell Technology has developed a solution using photonics to extend memory beyond the chip. This approach, demonstrated by Marvell's Photonic Fabric Business Unit, can improve efficiency in data centers by reducing wasted computational power. Preet Virk and Andy Mills spoke about this at SC26, highlighting how photonics and shared memory pools can alleviate these bottlenecks. Memory transactions now make up more than 80% of scale-up network traffic, and a shared pool would minimize data movement entirely. Virk explained that optics can also change the energy dynamics, reducing the energy required for data transmission. As inference spreads, Mills sees shared pools as the practical solution to these memory challenges, as recomputation can be avoided. The interview was part of theCUBE's coverage of SC26, with Penguin sponsoring this segment.",
  "summary": "Memory bottlenecks are emerging as one of the costliest problems in AI infrastructure as inference overtakes training. GPUs sit idle waiting on data, while high-bandwidth memory, or HBM, stays bolted to processors that cannot share it. Photonics is shaping up as one way out. Marvell Technology Inc. has been pairing optics with photonic memory to […] The post AI’s hidden energy tax: More than 80%…",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}