AMD acquires Taalas to boost inference performance by etching models in silicon

AMD has entered into a deal to acquire Toronto-based chip startup Taalas, aiming to bolster its capabilities in the rapidly expanding AI inference market. The announcement was made on August 6, with the financial details remaining undisclosed. The acquisition will integrate Taalas's unique inference technology and engineering team into AMD's Artificial Intelligence Group, subject to regulatory approvals and closing conditions.

Founded in 2023, Taalas has developed processors that integrate artificial intelligence models directly into silicon, effectively turning model weights and associated dataflows into dedicated hardware. This approach targets one of the primary constraints in modern AI computing: the movement of vast amounts of data between memory and processors. By eliminating this bottleneck, Taalas attempts to address the limitations imposed by memory bandwidth, even when substantial computational capacity is available.

The company's first technology demonstrator, HC1, runs Meta's Llama 3.1 8B model and is fabricated using TSMC's 6-nanometre process. The 815-square-millimetre chip contains approximately 53 billion transistors and can generate about 17,000 tokens per second for an individual user. While these performance figures were produced by Taalas, they should be considered vendor benchmarks rather than independent measurements.

Taalas's architecture uses a hard-wired mask-ROM fabric alongside SRAM, supporting features such as key-value caching, model adaptations, and fine-tuning. The processor's design reduces the cost and time involved in adapting the processor to another model, with only a small number of metal layers needing modification. Its development process aims to convert model weights into deployable hardware within roughly two months.

AMD plans to integrate Taalas's technology into its accelerator roadmap, utilizing it alongside Instinct accelerators, EPYC server processors, networking hardware, and AMD's ROCm software platform. This approach creates systems that assign different AI workloads to various types of silicon, reflecting the growing distinction between AI training and inference.

The strategy also reunites AMD with engineers who previously held roles at AMD, including co-founder and chief executive Ljubisa Bajic, who previously worked on AMD CPU-GPU designs.