AT&T Slashes AI Costs by Adopting Model Routers and Open Source

AT&T has reduced the costs associated with coding and other advanced AI tasks by up to 56%, according to The Information. The telecommunications giant achieved this by employing tools that direct employee queries to more cost-effective models when necessary, resulting in a mere 2% drop in AI performance, as cited in an interview with Mark Austin, AT&T's vice president overseeing AI usage.

These tools, known as LiteLLM model routers, assess task complexity and then decide whether the task can be handled by a more budget-friendly AI model.

AT&T intends to maintain its employees' spending on AI models from Anthropic and OpenAI constant by increasing the use of open-source or open-weight models. The company aims to boost the proportion of employee queries powered by open-source models from the current 40% to between 60% and 70% over the next few years, as reported. These models include Nvidia's Nemotron, Meta's Llama, and Google's Gemma.

While AT&T has not adopted open-source models from Chinese firms DeepSeek and Moonshot, the company is considering the potential risks associated with using them, according to The Information. Austin noted that open-source models are currently six to 10 months behind frontier models in capabilities, but the gap is closing, and these models are "just as good or better" than older models from Anthropic and OpenAI.

The surge in AI costs has prompted companies to adopt more efficient practices, as reported in June. Factors contributing to the rise in costs include the transition from chatbots to agents, which demand more computing power, and the shift from flat subscriptions to token-based billing by AI labs. New tools are being developed to tackle the issue of AI costs, and the era of "tokenmaxxing" or pushing employees toward the most powerful AI models and maximum usage is ending after two years of unchecked growth, according to PYMNTS.