Cerebras CS-4 rack systems juice their dinner-plate-sized AI chips for every last drop of AI perf

Cerebras has unveiled its new Wafer Scale Engine (WSE) and Nexus rack systems, aiming to boost memory bandwidth for AI inference. The company's dinner-plate-sized AI accelerators are already 1,000 times faster than Nvidia or AMD's best GPUs, with a memory bandwidth of 21.6 petabytes per second. The newly announced WSE-3T, or "Turbo," promises twice the compute, memory fabric, and I/O bandwidth of the previous generation, while using the same process tech, wafer area size, transistor count, core count, and SRAM capacity.

The main innovation here seems to be related to power delivery, which is reportedly so efficient that Cerebras can push twice the power through the chip, enabling higher operating frequencies and faster token generation. However, some experts are skeptical about the claimed performance figures, suggesting they rely heavily on sparsity, which doesn't benefit LLM inference.

Cerebras has partnered with Amazon Web Services and AMD to offload compute-intensive tasks to their respective accelerators, allowing Cerebras' chips to primarily function as decode accelerators. The company's new rack-scale compute architecture features modular design with a "backpack" form factor, enabling easier deployment, maintenance, and upgrades.

Each CS-4 rack can be equipped with up to three backpacks, housing the accelerators, power delivery, and cabling. Cerebras claims that its more efficient power delivery allows it to push twice as many watts through the chip, with a single rack consuming around 46 kW. While this may seem like a considerable amount of power for a liquid-cooled machine, it is relatively conservative compared to the 240 to 250 kW rack systems from AMD and Nvidia expected in the coming years.