Fireworks says you’re wasting money on AI unless you’re using all the models
The case for managed inference, or automatically routing queries to the right model for the right job
Fireworks AI, a managed inference provider, advises companies to reconsider spending money on AI unless they are utilizing multiple models. Rob Ferguson, the VP of Fireworks AI, argues that GPUs are expensive and it's a waste of money to keep them underutilized. He explains that Fireworks supports on-prem AI but recommends against it due to economic reasons.
Instead, Fireworks recommends using a portfolio of models, even if it seems counterintuitive to use multiple models rather than a single best model. The company's Nexus platform and API route inference tasks to the optimal model based on task difficulty and model affinity, such as Kimi being slightly better at JavaScript than Python.
Fireworks' engineering team, with experience in PyTorch, specializes in performance optimization for training and inference. The company emphasizes the importance of tracking cost per task rather than just token costs, as model rework is often the biggest cost in AI. Companies should also consider the value of their data, as models are becoming memorizers and should be compensated for using it.
Finally, Fireworks claims that its model routing activities can lead to at least a 50% reduction in costs for customers.
Written by urgent.news from The Register Science's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
This story
This is one outlet's version. Read the fullest account.
