Small Models Have Arrived — And They Change the Economics of Everything

Small, affordable models have arrived, fundamentally altering the economics of everything. GPT-5.6 Luna, priced at $0.20 per million input tokens and $1.20 per million output tokens on OpenRouter, costs $0.02 per million cached input tokens due to prompt caching. This price drop makes complex, multi-step agent workflows far more economical, costing $0.10 instead of $1. This represents a phase transition in what can be built.

Three structural shifts have enabled small models to become cost-effective and capable:

1. Mixture-of-Experts (MoE) became the default architecture for major releases in 2026. This approach allows models to have the capacity of a 300B-parameter model while incurring the inference cost of a smaller model, such as GLM-5.3-Flash (18B/320B) or Qwen3.8-Flash (6B/125B). These models achieve a 95% sparsity rate, drastically reducing costs.

2. Inference optimization has collapsed token costs. Luna's pricing structure reveals that the margins lie in standard rates ($0.20/$1.20 per million tokens) and huge cached input ($0.40/$1.80 per million) prices, while prompt cache hits and batch API calls are significantly cheaper ($0.02/$0.04 and $0.10/$0.60 per million, respectively). Batching inference workloads can reduce input costs to as low as $0.10 per million.

3. A fast and cheap archetype has its own Pareto frontier. Artificial Analysis's chart shows that GLM-5.3-Flash and GPT-5.6 Luna sit as the best open-source and proprietary models, respectively, balancing quality and cost. The old assumption that small models are inferior has been overturned—choosing the smallest model that meets evaluation requirements is now economically viable, as the cost difference between tiers can be an order of magnitude.

These changes unlock concrete workloads that make unit economics for consumer AI applications sensible. For example, a daily personalized news site saw a difference of $1/month versus $0.10/month with Luna, enabling healthy margins. In consumer products, AI features can now be delivered without inference costs devouring budgets.

Most business operations fall into the "token spewer" category—responsive tasks like sales follow-ups, customer support, code review, spec drafting, and status updates. GPT-5.6 Luna can handle 100 tokens per second, making it viable for real-time agent loops that were previously too slow or expensive. This enables multi-step agent workflows, where the cost of each LLM call (now $0.01–$0.05) allows for 10-step reasoning chains, verification loops, and retry logic. This shift changes how agent systems are designed, prioritizing correctness over cost.

While small models have arrived, they are not frontier models. GPT-5.6 Sol still outperforms them on complex reasoning and novel problem-solving. MoE models face a cold-start problem, requiring router networks to learn which experts to activate, leading to suboptimal performance on novel tasks.