Small Models Have Arrived

Small and fast models have gained significant traction recently. GPT-5.6 Luna, in particular, has been impressively capable, fast, and smart. However, its cost is a major concern. Running complex research threads or even searching through thousands of emails can quickly inflate the API cost. Despite this, there's a new option available that could change the game.

Investors have noticed this shift, questioning why there aren't more consumer AI companies. The answer lies in the high token costs associated with AI inference, making it difficult for many businesses to incorporate AI into their products without incurring substantial capital expenditure. For instance, building a personalized daily news site using previous generation models like Sonnet would cost around $1 for basic functionality. However, with Luna, the cost drops to about $0.10, making it much more feasible.

This trend is particularly relevant in the business world. Many entrepreneurs, like Segment co-founder Peter, find that 95% of their work involves tasks such as managing calls, nudging people, and tackling day-to-day issues. These are areas where fast, cheap, and good-enough models could be highly beneficial. While making such models a reality will require significant advancements in harnesses, prompt injection safety, roles, and permissions, the potential benefits are undeniable.

The author expresses confidence that businesses will eventually adopt these models, much like Amazon and Netflix have already done. As experimentation with small models continues, they encourage others to share their experiences and insights.