Tokens too cheap to meter

The cost of utilizing machine learning intelligence is rapidly decreasing by several orders of magnitude each year and shows no signs of slowing down. In the next year or two, we can expect Large Language Models (LLMs) to be integrated into every aspect of computing as infrastructure, not just as a standalone product. By 3-6 years, LLMs are likely to run locally on standard hardware with the same frontier-quality.

Soon, quality and accessibility will become the primary constraints for AI usage, rather than the sheer number of tokens. Evidence supports these claims.

AI models can be proprietary, like GPT-6 Astra, or open weight, such as GLM-5.3-flash. Open weight models can either be hosted, like Z.ai, or run locally. Typically, models meant for local use are smaller, like Muse Glimmer or Qwen3 Coder. Improvements in one area, such as model size, do not necessarily correlate with improvements in the other areas, like intelligence.

Graphics Processing Units (GPUs) are becoming exponentially more efficient with each generation. The graph provided demonstrates that the Y-axis represents the power efficiency of the GPU itself, with larger numbers indicating greater efficiency. The graph uses a logarithmic scale, where a straight line signifies an exponential increase in efficiency.

In this case, the logarithm is 1.3, meaning efficiency doubles approximately every two years. This efficiency increase surpasses the improvements seen since Moore's Law in the 1960s.

The cost to complete a task with a model is decreasing sharply over time. Models are generally priced per-token, where a token is a fragment of a word, approximately 1.5 tokens per word. For every token a model reads and outputs, the model provider (e.g. Anthropic or OpenAI) charges a fixed fee. The cost per token varies between models, with smaller models often being cheaper per token but requiring more tokens overall for the same task due to increased processing and corrections. The focus here is on the cost to complete the task from start to finish.

The chart below shows the current state of the Pareto frontier of cost/task. This chart represents the best tradeoff achievable, not just the best in a single category. In 2026, models are smarter and cheaper on a per-task basis compared to the start, middle, and end of 2025. If you draw a horizontal line at any task on the Y-axis, the cost to perform it at the end of 2025 is cheaper than at the beginning; similarly, if you draw a vertical line at any point on the X-axis, models can achieve more for the same cost.