Meta's new MTIA 400 chip has a split personality: Training AI and serving ads

Meta's new MTIA 400 chip serves a dual purpose: training AI and serving ads. This custom AI accelerator, teased earlier this year, was unveiled at the annual Hot Chips semiconductor development conference. While the MTIA 400 can handle some inference duties, it's primarily designed for running ad recommender systems, which are crucial to Meta's revenue.

The MTIA 400 combines LLM training and DLRM inference, two workloads with distinct performance demands. LLM training is extremely compute-intensive, while DLRM inference is memory-bound, leading to a lot of FLOPS going idle. However, Meta's financial investment in compute makes it beneficial to have chips that can perform both tasks, even if one use case is highly profitable.

The MTIA 400 is based on a heterogeneous multi-die architecture, built from various chips each with its own job. It features two compute dies, two I/O dies, and an SoC die for host connectivity and workload orchestration. Broadcom's XPU technology is likely used in the chip's design. The MTIA 400 boasts 12 petaFLOPS of MXFP4 compute at 1.7 GHz, with eight 36 GB HBM3e stacks delivering 288 GB of memory and 9.2 TB/s of bandwidth.

These specifications make the MTIA 400 slower than Nvidia's and AMD's latest chips, particularly in inference tasks. Meta plans to release an inference-optimized version, the MTIA 450, with double the memory bandwidth next year, and the MTIA 500, increasing memory bandwidth by 50 percent and doubling the compute chiplets, in 2027.