What GLM-5.3 Flash running on Chinese hardware actually means

Z.AI has recently confirmed that their latest model is running inference on Chinese-made hardware. However, Western companies still hold a significant advantage in this area. The focus of this report is on HiSilicon, the most competitive chip manufacturer from Huawei, despite other AI hardware manufacturers existing. It's important to note that Z.AI didn't specify the chipmaker or publish throughput or power figures, making independent verification of their claim impossible.

The HiSilicon 910c series, currently the most advanced, features 96GB HBM2e memory with two compute dies, achieving approximately 1.6PFLOP/s of INT8 compute and 3TB/sec memory bandwidth at around 600W. This is about 60% as fast as Nvidia's H100, which was released four years ago. The next-generation 950-series doesn't offer substantial improvements in compute capabilities, but it does use domestically produced HiZQ/HiBL HBM memory.

The two variants, 950PR and 950DT, differ mainly in memory type - slower HiBL for prefill and faster HiZQ for decode. While supporting more quantization types, including FP8, Chinese hardware still lags behind Nvidia's Jalapeño inference chip, which outperforms Nvidia's GB300 by 1.5-1.9x per watt.

A major limitation for Chinese hardware is the lack of EUV (extreme ultraviolet) fabrication technology, which is crucial for producing smaller and more efficient chips. Without this technology, Chinese manufacturers cannot produce chips below the 7nm fabrication size, leading to thermal inefficiency and limited scalability. Additionally, export restrictions on HBM memory further complicate the situation.

Despite these challenges, Chinese manufacturers are focusing on building a large quantity of underpowered hardware to compensate for the performance gap, which may result in slower overall inference speed. Power consumption is also a concern, with the 910c consuming around 600W compared to the more powerful Rubin part's 2000W.