Who’s Afraid of Chinese Models?

The keyword "Chinese" pops up again as this past weekend saw heated debate on X about the implications of Kimi K3, an open weights model originating from China. Analysts are grappling with the fact that marginal costs are making a strong comeback, both in terms of short-term implications of free models and long-term industry structure.

A common misconception is that open weights models are cheaper – free, even – as they require no research and development expenses. However, the "free" in this context actually refers to the minimal research and development costs, while the cost of goods sold (COGS) is real for AI. Running inference on a model, including those like Kimi or Fable, incurs costs, and the amount spent on inference is directly correlated to revenue.

Open weight models, such as Kimi K3, are not free to serve, costing $3 per million input tokens and $15 per million output tokens. While this is cheaper than other models, it may not be the most accurate measure. Token efficiency varies between models, with some like Kimi using significantly more tokens than others, making its price advantage questionable.

Moreover, the fungibility of tokens is limited, as intelligence constructed from tokens is not interchangeable. The cost of intelligence is influenced by factors like model footprint, inference efficiency, memory efficiency, serving efficiency, and token efficiency. As we inch closer to a stage where intelligence becomes a commodity for economically beneficial tasks, the COGS for intelligence becomes a crucial factor in determining the cost-effectiveness of different models.