Tiny's new AI workstation is anything but — packing in 16 TB SSD array and 128 GB GPU memory, but with an eye-watering price tag to boot

Tiny Corp has introduced its latest AI workstation, the tinybox Red Kimi, which boasts an impressive set of specifications. The machine packs four Radeon AI Pro R9700 GPUs, each with 32 GB of GDDR6 memory and 640 GB/s bandwidth, into a single unit, along with sixteen 1 TB NVMe SSDs. The system is built on Tiny Corp's G5 platform and features a 32-core AMD Epyc Genoa CPU, along with 32 GB of DDR5 system memory.

The standout feature of this workstation is its storage solution, consisting of a RAID array formed by the 16 drives, rated at 238.4 GB/s. Tiny Corp claims that its software, tinygrad, now includes a driver that lets the AMD GPUs control the NVMe drives directly, bypassing the CPU. This unique approach enables the workstation to run Moonshot AI's 2.8-trillion-parameter Kimi K3 model, which is claimed to achieve a speed of 40 tokens per second.

However, the $30,000 price tag attached to the Red Kimi raises questions about the practicality and affordability of such a high-performance machine. The GPU memory alone is not sufficient to hold the entire model's weights, so the SSD array plays a crucial role in providing the necessary storage. Yet, the claim of 40 tokens per second depends on the effective utilization of the GPU memory and the efficiency of the SSD's storage capabilities.

Comparatively, the Red Kimi's price is on par with Apple's Mac Studio and Nvidia's DGX Station, but these alternatives lack the same level of storage capacity and performance. Other contenders, such as AMD's Threadripper Halo Station, are not expected to be released until 2027 and are estimated to cost around $150,000.

While the Red Kimi offers the cheapest way to maintain Kimi K3's native precision on a single desk, its performance depends on the efficiency of caching, quantization, and decoding techniques used. Until independent testing confirms the claimed 40 tokens per second, the Red Kimi remains a promising but unverified solution in the field of AI workstations.