Why compute might get 10x+ more expensive in coming years

In the coming years, compute costs may experience a 10x increase or more, according to recent wire material. Anthropic's revenue has more than doubled year-over-year, with projections suggesting it could reach $100–150 billion by the end of next year. To achieve this growth, labs would need to significantly increase their margins, compute prices, or allocate a larger share of compute to inference.

Several factors are driving up compute costs: Anthropic's margins have improved from 40% to over 80%, Spot prices for compute have risen by 40%+, and a significant portion of OpenAI's 2024 compute spend was allocated to inference (50% or more). However, labs prefer not to spend an increasing fraction of their compute on inference, as it suggests AI progress has stalled.

For margins to dominate, they would need to reach around 90% by the end of next year. This seems unlikely, but AI lab revenues could continue growing at an astonishing pace. The key conclusion is that as AI models become smarter, they will better monetize the same amount of compute. For instance, if a human-level software engineer could run on an H100 equivalent, it should cost over $250,000 per year, which is 15x the current spot price.

If AI models become 10 million times more efficient, the marginal value of a software engineer would decrease, and the H100 would not generate 15x more revenue than today. However, the argument that high-skilled immigration does not decrease wages in the long run suggests that the marginal value of labor (and thus the marginal price of compute) should remain exceptionally high.

In such a world, as top models become more efficient at monetizing compute, it will become increasingly difficult for competitors to catch up. If by 2028, the price of compute is 15x higher than today, it will be much harder for businesses without significant revenue to compete for compute against the leading labs. The Alchian–Allen effect suggests that labs will be able to charge a large premium if they can train a model that better utilizes scarce compute resources.

As compute costs rise, popular AI applications may become more expensive, and businesses will need to adapt to this new reality.