Chipmaker Positron nabs $875M to speed up inference with consumer-grade memory

Chipmaker Positron AI Inc. has secured $875 million in funding to develop faster inference appliances using consumer-grade memory. The company, led by investors including NEA, Atreides Management, and Jim Clark, now has a valuation of $5 billion, a fivefold increase from February.

Inference, the process of using trained models to generate outputs, often requires moving data between computing circuits and memory. Memory bandwidth, the speed at which this data transfer occurs, is crucial for performance. Server-grade graphics cards use high-bandwidth memory (HBM), but its supply is limited, and so are the interconnects needed to integrate it with computing circuits.

Positron bypasses this supply chain issue by using LPDDR5X, a memory type used in smartphones, as a substitute for HBM. Despite its lower memory bandwidth, Positron claims its appliances can utilize over 90% of LPDDR5X's throughput, compensating for the performance gap. The flagship system, Titan, comes with up to 18.4 terabytes of LPDDR5X, capable of running large language models (LLMs) with 32 trillion parameters and a context window of 10 billion tokens.

At the heart of Positron's Titan appliance is a custom chip called Asimov, built around a systolic array of computing modules with co-located memory. This design allows for faster activation function computations and more efficient use of LLM weights. Multiple Titan systems can be linked together in clusters, with up to 16,384 accelerators.

While Titan and Asimov are not yet in production, Positron expects to begin mass production in the second half of 2027. The company aims to achieve up to 26 times more tokens per dollar compared to Nvidia's Blackwell GB300 NVL72 appliance, according to their simulations. CEO Mitesh Agrawal stated that the recent funding will enable them to meet the anticipated demand.