Inference chip startup Etched raises another $700M at $21B valuation

Inference chip startup Etched Inc. has announced raising an additional $700 million in funding, bringing its total valuation to an impressive $21 billion. Led by investment firm Jane Street, the round also saw participation from Kleiner Perkins, Sequoia, Andreessen Horowitz, Tiger Global, Bain Capital Ventures, and several other investors. This substantial increase makes Etched one of the most valuable AI chip startups in the market.

Etched was founded in 2021 and initially focused on designing chips tailored for specific artificial intelligence models. However, the company later shifted its focus towards creating inference accelerators that can support various model architectures. In early 2023, Etched's first prototype chip was manufactured using Taiwan Semiconductor Manufacturing Co.'s facilities. The company quickly gained a customer in Jane Street, which installed an Etched-powered rack in its data center the following month.

In addition to the funding round, Etched has also built a two-megawatt AI cluster in its San Jose, California headquarters. This cluster serves as a testing ground for prospective customers to evaluate the company's chips. The performance of AI accelerators is closely tied to their operating temperature; the lower the temperature, the more calculations can be performed per second.

One of the key features of Etched's chips is their math blocks, or math-optimized circuits, which operate at less than half the voltage of most AI accelerators. This voltage optimization results in multiple times the FLOPs density compared to competitors, indicating superior performance. FLOPs density refers to the number of floating-point operations a processor can execute per second.

Etched's chips are optimized for two main types of mathematical operations in large language models: dot product calculations and matrix multiplications. Dot product calculations transform two rows of numbers into a single number, powering the attention mechanism of transformer architectures. Matrix multiplications involve mathematical structures that resemble spreadsheet tables filled with numbers, and they are used in various model components, including LLM attention mechanisms.

The startup's silicon is shipped as part of custom-designed racks, featuring an internally developed interconnect that significantly speeds up communications between chips. This interconnect can perform tasks that rival chips take 4,000 milliseconds to complete, doing so in just 700 milliseconds. Other unique components in Etched's appliances include custom cold plates for heat dissipation and a proprietary VRM (voltage regulator module) that optimizes the flow of electricity.

Etched co-founder and CEO Gavin Uberti mentioned that it took three years to develop their first rack from scratch, with the next one being ready to ship much faster.