[Insight] Beyond HBM: SK hynix Maps the Next AI Battleground in Optical Interconnects

SK hynix has unveiled plans to incorporate optical interconnects, or CPO, into its AI infrastructure beyond HBM, addressing the widening gap between computing power and data movement. This study, published in Nature Electronics, highlights the critical challenge of bandwidth bottleneck as AI clusters expand. The research outlines a roadmap for co-packaging optics to facilitate high-speed connectivity between processors, memory, and large-scale computing systems.

The researchers aim for systems to deliver over 100 Tb/s of bandwidth per node with low power consumption and latency. By integrating optics closer to processors, they intend to maintain high bandwidth density and energy efficiency even as connectivity expands. This evolution is reminiscent of HBM, which improved AI computing efficiency by bringing memory closer to processors.

CPO, however, seeks to scale this principle by reducing the distance traveled by high-speed signals and using optical links for longer connections. The ultimate vision extends optics into memory architecture, enabling processors to access shared memory resources through high-bandwidth optical connections even if they are physically separated.

This approach could prove crucial as AI models continue to expand. SK hynix, a major beneficiary of the generative AI boom, acknowledges that faster computing chips alone cannot eliminate every bottleneck. They emphasize the need for co-design, integrating memory devices, controllers, optical components, processors, and advanced packaging technologies to work together harmoniously.