How well can we measure open model inference share?
Our first open model economy research note for paid subscribers.
The Interconnects Dashboard, a compilation of open model adoption and performance data from the web, provides valuable insights into the open model inference space. However, the metrics used to measure this market remain highly imprecise due to the reliance on noisy platform metrics, startup marketing data, and imperfect public evaluations. The dashboard aims to offer free, valuable data that can be used to better understand the industry's trajectory.
Open model startups such as Baseten, Fireworks, and Together AI serve as key indicators for the growth of open model inference. These companies' fates are closely tied to the success of open model inference as a category. The latest data comes from slides presented in a talk on the open model economy, which showcase the exponential growth in token volume for these startups over time.
For instance, Fireworks reported serving more than 40 trillion tokens per day to customers' customized models, with a projected increase to well over 100 trillion tokens in the upcoming period.
To put these numbers in context, token volumes from closed model providers and hyperscalers like Amazon Bedrock and Azure's model garden are necessary for a comprehensive comparison. Unfortunately, these companies have not broken down their usage by model category, making it difficult to gauge their impact on the open model inference market.
Nonetheless, Google's public disclosures on platform token volumes and API usage offer a valuable reference point for comparing open model inference providers against closed model players.
Written by urgent.news from Interconnects's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
