AMD inches closer to its goal of making AI suck less ... energy

AMD appears to be making significant strides in reducing the energy consumption of AI systems, with the house of Zen estimating its systems to be four times more efficient in 2026 compared to 2024. After the launch of the MI300X datacenter GPU in 2023, AMD has been focused on squeezing additional FLOPS per watt, enhancing memory and fabric performance, and optimizing software to keep pace with competitors like Nvidia.

The company has introduced 4-bit floating point data types, new memory technologies, faster interconnect speeds, and integrated rack-scale systems. This efficiency boost is primarily due to AMD's ability to scale AI workloads across its 72 GPU accelerators (MI455X), delivering up to 7.7x to 15.4x higher floating point performance, 2.25x more HBM, 4.4x faster memory, and 4x interconnect bandwidth.

While Nvidia has also ventured into rack-scale architectures, AMD employs a distinct approach for calculating efficiency, taking into account max achieved FLOPS, memory, and interconnect bandwidth separately for training and inference. AMD's projected efficiency gains could enable two Helios racks to match the performance of 570 racks from 2024, potentially allowing customers to deploy 20 times more compute with the same power consumption.